The experiment most innovation teams start with (and get wrong)

They're cheap, quick to set up, and feel productive. But most teams approach them completely backwards, asking hypothetical questions that generate opinions instead of insights.

What's a Customer Conversation test?

It's a structured interview focused on understanding the real jobs customers are trying to do, the pains they experience, and the gains they're seeking. Not what they think about your idea, but what drives their actual behaviour.

When done properly, customer conversations reveal the underlying motivations that quantitative data can't capture. When done badly, they create false confidence in ideas that will never work.

The difference lies in what you ask - and more importantly, what you don't ask.

How a software company discovered their £5M assumption was wrong

An enterprise software company we work with was developing an AI-powered project management tool. Their initial market research suggested strong demand for "smarter resource allocation" and "predictive timeline management." The features tested well in concept surveys.

Before building the platform, they decided to run customer conversations with 15 project managers at their target companies. Most teams would have asked: "Would you use AI to help with resource allocation?" or "How valuable would predictive timelines be?"

Instead, we helped them dig into actual behaviour and pain points:

What they asked: "Walk me through your last project that ran over budget or missed a deadline. What happened?"

What they discovered: Project managers already knew which projects would be problematic, they just couldn't do anything about it. The issue wasn't prediction; it was politics. Senior stakeholders kept changing priorities mid-project, but project managers had no authority to push back.

What they asked: "Show me how you currently track resource allocation. What's frustrating about this process?"

What they discovered: The pain wasn't tracking resources, it was getting accurate time estimates from developers who were incentivised to underestimate. Most project managers had elaborate workarounds involving "developer multipliers" and hidden buffer time.

What they asked: "Tell me about a recent project that went really well. What made the difference?"

What they discovered: Successful projects had dedicated, protected teams with minimal context switching. The value wasn't in optimising resource allocation across multiple projects, it was in reducing the number of concurrent projects per person.

Based on these conversations, the company completely pivoted their approach:

• Scrapped the AI prediction features that addressed symptoms, not causes

• Built a tool focused on project portfolio management and team protection

• Created features that helped project managers communicate resource constraints to stakeholders

• Developed "focus metrics" showing the cost of context switching

The repositioned platform:

• Reduced development time from 18 months to 8 months

• Generated £2.3M in revenue within the first year

• Achieved 73% customer retention vs. predicated average of 45%

• Created clear differentiation in a crowded market

Without proper customer conversations, they'd have built yet another AI-powered project management tool that solved the wrong problem.

How to run Customer Conversations properly

Most teams get customer conversations wrong because they ask about solutions instead of problems, or hypotheticals instead of real experiences.

1. Focus on past behaviour, not future intentions

Don't ask: "Would you use a tool that predicts project delays?" Ask: "Tell me about the last project that ran over schedule. What were the warning signs?"

Don't ask: "How much would you pay for better resource management?" Ask: "What's the most expensive resource allocation mistake you've made recently?"

2. Dig into actual workflows, not ideal scenarios

Don't ask: "How do you currently handle resource planning?" Ask: "Walk me through exactly what you did yesterday when the client requested that urgent change."

Don't ask: "What features would be most valuable?" Ask: "Show me the spreadsheets, tools, and workarounds you use now. Which parts are most frustrating?"

3. Understand the job behind the job

Don't ask: "What are your biggest challenges with project management?" Ask: "Tell me about a time when a project failure reflected badly on you personally. What happened?"

Don't ask: "What would success look like?" Ask: "Describe a recent win that made you look good to your boss. How did you achieve it?"

4. Look for emotional language and energy

Pay attention to when people get animated, frustrated, or excited. These emotional responses reveal what actually matters, not what they think should matter.

Common mistakes that kill insights:

• Asking about your solution instead of their problem

• Leading with features rather than understanding jobs

• Accepting surface-level answers instead of digging deeper

• Talking more than listening

• Asking hypothetical "would you" questions instead of "tell me about when you" stories

Try this next week

Pick three potential customers for your current innovation project. Instead of asking what they think about your idea, ask them to walk you through a recent situation where they experienced the problem you're trying to solve. Focus on understanding their actual behaviour, not their stated preferences.

You'll likely discover that the real problem is different from what you assumed, and the solution needs to address deeper underlying issues.