Test Complex Tech Without Writing a Single Line of Code
Jacob Dutton
20 Mar 2025
You don't need to build large language models, algorithms, or complex systems to test if they'll work. You just need to make users think they're using technology while humans secretly do the work behind the curtain.
What's a Behind-the-Scenes Test?
It's a test where your users think they're interacting with a working product, but humans are manually performing the functions behind the scenes.
Think of it as a magic trick; users see an interface, but the "magic" is actually humans frantically working to deliver the experience.
It's particularly useful if you need to validate complex ideas with expensive development pricetags. AI means we're running a LOT of these.
How a logistics company saved a year of development
A big logistics provider we worked with wanted to create an AI system that would optimise delivery routes in real-time based on traffic, weather, and package priority. The estimated development investment was 14 months and £3.2M.
Before writing a single line of code, we ran a Behind-the-Scenes test with them. We created:
A simple driver app that appeared to provide AI-optimised routes
A basic dispatcher interface that showed route changes
A team of analysts who actually calculated the routes manually
We tested this with 8 drivers over 3 weeks.
The concept worked brilliantly; drivers followed the routes and deliveries were 22% more efficient. But the drivers kept ignoring certain types of recommendations, like cutting through residential areas or using service roads.
This human behaviour would have undermined our entire model.
Instead of spending a year building a system drivers wouldn't fully use, we redesigned the experience to incorporate driver preferences and local knowledge. The revised system launched in 7 months, cost £1.4M less than planned, and saw 94% driver compliance.
How to run a Behind-the-Scenes Test
To run this test, you'll need:
A simple front-end interface
A manual process to deliver the service
People who can act as the "intelligence" behind the scenes
1. Create a believable façade
Design the simplest possible interface that makes your service feel real. It doesn't need to be pretty—it just needs to be believable.
2. Set up your behind-the-scenes operation
This is where the real work happens. Create processes for how your team will:
Receive user inputs
Process them manually
Deliver outputs that appear automated
Handle unexpected requests
3. Launch with real users
Tell users they're testing a "beta" or "prototype" version. This sets appropriate expectations while still making the experience feel real.
4. Watch what happens
The most valuable insights come from:
How users interact with the "system"
What they expect it to do versus what you planned
Where they get confused or surprised
How they respond to the outputs you provide
5. Gradually reduce manual intervention
If the test is successful, start automating pieces of the process to scale the operation.
Common mistakes to avoid:
Not having enough people behind the scenes (things get slow, breaking the illusion)
Creating interfaces that are too complex to maintain manually
Setting unrealistic user expectations about response times
Missing the opportunity to learn from unexpected user requests
Try this next week
Think of a complex feature your team wants to build. Create a simple interface that simulates how users would interact with it. Have team members manually provide responses through the interface. Test with 3-5 users and observe their reactions.
You'll be shocked by what you learn about your assumptions.
More articles
Explore more insights from our team to deepen your understanding of digital strategy and web development best practices.
