Email metrics mislead innovation teams because they measure interest in your content, not demand for your product. A high open rate shows your subject lines work. A high click rate shows your headlines work. Neither tells you whether people have a problem you can solve or will pay to solve it.
That matters, because many teams use email campaigns as a demand test. They send a sequence explaining the idea, measure engagement and treat good numbers as proof. We call this a Message Engagement test. Used well, it’s useful. Used as proof of demand, it misleads.
What is a message engagement test?
You build an email list, write a sequence of messages that introduces your idea over time, and track how people engage. The theory is that the people who consistently open, click and reply are your future customers.
What email metrics actually measure
Email engagement tells you whether your messages resonate with people who are already interested in the topic. If you’re building productivity software and emailing people about productivity, high engagement means you can write good productivity content.
That’s useful to know. But it isn’t proof of demand.
Think about what drives email behaviour. People open emails because the subject line catches their eye. They click because a headline promises something interesting. They reply when something sparks a thought or a question.
None of that requires them to have a problem your product solves. Someone can find your productivity content fascinating and still be perfectly happy with how they work now.
Open rates have also become less reliable. Since 2021, Apple Mail’s privacy features can record an email as opened even when nobody has read it, so a strong open rate may be partly an illusion.
What a 34% open rate actually predicted
Teams often chase engagement numbers without linking them to business outcomes. A 40% open rate feels like a win next to industry averages, but what does it tell you about how many people will buy?
We worked with a team building financial planning software whose email campaign looked impressive: a 34% open rate, an 18% click rate and 127 replies. When they launched their beta, only 9% of those engaged subscribers actually tried the product.
Their product wasn’t the problem. Their campaign had attracted people interested in financial content, not people actively looking for a financial planning tool.
We’ve seen the same pattern elsewhere. In our £185K pop-up store example, 412 email sign-ups were mostly people hoping for a discount code.
Is your business case built on engagement numbers?
If your evidence for a new idea leans on open rates, clicks or sign-ups, it’s worth a conversation about what would count as real proof of demand. Miles, Managing Partner at Future Foundry, can walk through whether an email test, or another business experimentation method, fits your situation.
Book a call with Miles here.
When email engagement tests work well
1. Testing messages with an audience you’ve already qualified
If you already know your audience has the problem you’re solving, an email campaign can show you which messages, features and timing land best.
2. Comparing different ways to position your idea
Send different versions of your pitch to similar groups and see which one gets stronger engagement and more follow-through.
3. Learning what people need to know first
A sequence of emails can show you how much context people need before your idea makes sense, and what information moves them closer to considering it.
What to measure beyond opens and clicks
1. Actions that take effort
Track actions that take more effort than a casual click. Someone who replies with a specific question about how your product would work shows much stronger intent than someone who clicks through to read more.
2. Unsubscribe patterns
A spike in unsubscribes after a particular email might point to a messaging problem. It might also mean you’re successfully filtering out people who never would have bought.
3. What happens when you offer a next step
Offer people who engage a demo, a trial, or a call, and track how many take it. That’s what bridges the gap between interest and real intent. Click Tracking helps you see exactly where people drop off.
Three ways to design a better email test
1. Target specific behaviours, not broad topics
Instead of “productivity tips”, target “people who regularly work weekends” or “teams that keep missing project deadlines”. Specific behaviours point to specific problems.
2. Test the problem before you introduce the solution
Start by checking whether people recognise and care about the problem you’re solving. Strong engagement with problem-focused emails tells you far more than engagement with product emails. Our Mental Mapping method is another way to find out which problems are worth solving.
3. Build in real commitment steps
Give people chances to do more than click, such as booking a call, completing a detailed questionnaire or requesting tailored information. These take more effort, so they signal stronger intent. We’ve written more on the commitment test that reveals phantom demand.
Try this: a three-email problem test
Design a short sequence that tests whether people recognise the problem before you mention your idea.
What to send:
Email 1: describe a specific challenge without mentioning your solution.
Email 2: explore what that challenge costs people and what it leads to.
Email 3: ask people to share their own experience of the challenge.
Measure engagement with these problem emails separately from anything about your product. If people engage strongly with the problem but not with your solution, the issue may be timing rather than your idea.
Should you use email to validate demand?
Not on its own. Email tests can give you useful insight under the right conditions, but they work best as one part of a wider testing approach.
Use them to test messaging with audiences you’ve already qualified some other way. To prove real demand, use tests that ask people to commit, such as a fake door test, a Commitment-to-Buy test, or an Advance Purchase.
Common questions about email metrics and validation
Do high email open rates prove demand?
No. A high open rate shows your subject lines work and that people are interested in the topic. It says nothing about whether they have the problem you solve or will pay for a solution. Email privacy features can also inflate open rates, so they’re less reliable than they look.
What’s a better signal than clicks?
Anything that takes real effort or commitment: a reply with a specific question, a booked call, a started trial or, best of all, a payment. The more effort an action takes, the more it tells you.
Are email metrics useless for innovation teams?
No. They’re useful for testing which messages work with an audience you’ve already qualified. The mistake is treating them as proof that people want your product.
Is your business case built on engagement numbers?
If your evidence for a new idea leans on open rates, clicks or sign-ups, it’s worth a conversation about what would count as real proof of demand. Miles, Managing Partner at Future Foundry, can walk through whether an email test, or another business experimentation method, fits your situation.
Book a call with Miles here.
An email engagement test is one method in business experimentation, the discipline of testing what customers actually do before you commit budget to a solution. Explore the Message Engagement test page in the Toolbox, along with 50+ other methods in our Experiment Toolbox.
