%20(1).svg)
Last week, we organised a roundtable with other members of The Fold to examine how to test new ideas. We talked about different ways to design experiments, our favourite tests to run, and shared common challenges. It was brilliant, and you should definitely attend the next one.
What was most interesting about the session was that, at a theoretical level, everyone agreed that testing is the best way to remove the most risk from ideas with the smallest amount of capital. But, at a practical level, almost everyone misunderstood how to sequence their experiments, the right level of test fidelity to aim for, and how much time and money to invest in them.
In case you're facing similar struggles, I thought it'd be useful to give you a mental model to help you design, plan, and execute more effective tests.
Why Testing Ideas is Crucial
Ideas without evidence are futile. If you're developing a new venture, product, or service within your organisation, it won't receive sponsorship, funding, or support unless you can demonstrate three things:
- It's desirable: people need it because it addresses a real unmet or underserved need.
- It's feasible: you understand how, both technically and organisationally, to deliver the solution.
- It's viable: you can capture value and benefit your organisation financially.
You have to have evidence for all three – not just one or two. Desirable things that can generate a return but you can’t build? That's impossible. Things that are buildable and desirable, but it's unclear if they'll create a return? That's fantasy. And things that are buildable that can generate a return, but no one wants? That’s pointless.
In our experience, this is why 90% of all innovation ideas fail – because companies don’t have a systematic method to reduce the risks in searching for new ideas. Usually, because they lack the time and necessary skills or are blocked by internal processes.
This leads to the development of ideas that nobody wants, that can’t be built or won’t generate a return.
And that represents a colossal waste of time, money, and effort.
As a corporate innovator, your primary role should be to place small bets on incubating a large number of ideas and gathering evidence for them before investing in ideas that test well and killing those that don’t.
1. Finding Problem-Solution Fit
At the outset of any new project, your aim should be to understand customer problems and how they might be solved with new, improved, or more efficient solutions.
Since the solution isn't yet clear, there's no point in focusing on feasibility and viability experiments. Your focus should be entirely on desirability.
At this stage, you might develop between 50-100 ideas through team ideation, running an organisational competition, or looking at emerging competitors, technologies, and investor trends.
Your uncertainty will be high at this stage, so keep your budget allocation low – no more than 15% – and use this to run some low-fidelity 'customer say' experiments. These are typically, though not always, qualitative. Here’s some of our favourites:
- Customer Interviews
- Day in the Life
- Discovery Survey
- Discussion Forums
- Storyboard
- Search Trends
- Card Sorting
- Traffic Analysis
- Speed Boat
- Paper Prototype
- Validation Survey
- Support Analysis
- Sales Force Feedback
- Data Sheet
- Boomerang
Here, you should keep spending relatively low while focusing on identifying customer pains, rapidly prototyping, and iterating on idea prototypes based on customer feedback. This will help make sure you and your team don't invest in ideas without sufficient evidence of customer interest.
2. Finding Solution-Proposition Fit
Hopefully, you'll have identified some desirable solutions to real customer problems (if not, you can stop, having only spent 15% of your budget and a few weeks of time), and you'll want to translate those solutions into feasible products that can be consistently reproduced and delivered.
You should have between 10 and 20 desirable solutions at this stage. But, given we're focusing on productising these solutions, you'll need to introduce more focus on feasibility while still maintaining focus on desirability as your ideas mature.
You'll have more confidence in these ideas than in the first phase, but your uncertainty should still be too high to invest more than 30% of your budget in testing them. At this stage, increase the fidelity of your artefacts and experiments to medium and shift from the qualitative world of 'customer say' experimentation to observing what customers actually do with your products by running experiments like:
- Online Ad
- Landing Page
- Clickable Prototypes
- Letter of Intent
- Explainer Video
- Split Test
- 3D Print
- Partner + Supplier Interviews
- Expert Stakeholder Interviews
- E-Mail Campaign
- 404 Test
- Feature Stub
- Link Tracking
- Social Media Campaign
- Product Box
- Brochure
- Life Size Prototype
Costs will naturally increase with the development of more sophisticated prototypes in this stage, but by carefully planning this sequence, you can make each iteration is based on actual customer behaviour data and justify the increased expenditure.
3. Proposition-Market Fit
Hopefully, by now, you'll have identified between 3-5 desirable solutions that can be feasibly productised (if not, you can stop and cut your losses at 45% of your total budget), and you'll want to find evidence of demand at scale for your proposition and proof that the business model is sustainable and well-received.
This final stage is about proving that you have a viable model for your new venture, product, or service so you'll need to introduce a viability lens while maintaining a focus on desirability and feasibility. Ideally, aim for a 33% focus on each.
Given the evidence collected in the first two phases, your confidence in these ideas should be relatively high, as should your corresponding budget allocation. We typically recommend around 55% of your total budget for this stage because you'll again want to increase the fidelity of your artefacts and experiments to high and shift from the smaller scale world of 'customer do' experimentation to examining whether you can actually acquire customers and generate revenue through experiments like:
- Single Feature MVP
- Mash Up
- Crowdfunding
- Presale
- Mock Sale
- Pop Up Store
- Wizard of Oz
- Concierge
- Referral Program
- Buy a Feature
The costs are highest at this stage, but if the previous stages of desirability and feasibility + desirability have been executed well, the risk is significantly reduced, and the likelihood of a successful launch increases.
Moving Forward
I hope this framework helps you design, plan, and execute more effective tests.
In the early stages, the key is to keep costs low while maximising learning about customer desirability. As ideas prove their desirability, more resources can be allocated to develop prototypes to test feasibility. Then, only the most promising ideas that have been validated in terms of desirability and feasibility receive the total investment required to test their viability in the market.
It's this sequencing that enables you to de-risk ideas at each stage. By requiring validation of desirability before testing feasibility and feasibility before testing viability, only the most promising ideas receive investment to reach the next stage. This filtering process allows for more discipline, as budget allocation is directly tied to the evidence of an idea's success potential.
P.S. If you'd like access to our experiment toolbox along with information on what they are designed to test for, the costs, setup time, run times involved, and the strength of evidence each provides, just drop us an email, and we'll send it to you.
