EXPERIMENTATION TECHNIQUE
The Removal Test
A one-question technique for killing wasted optimization before it starts. Straight from Dr. Simon Jackson (Cherto) at our Adasight × Cherto webinar, this playbook breaks down a single, repeatable move: instead of asking how to improve something, ask what happens if you removed it entirely. Includes two real examples where the test gave opposite and equally valuable answers, plus how to use it as a pushback tool before a team over-invests in a complex idea.
The Cost of Not Asking
A team at Booking.com spent nearly a year on one part of the booking flow: new copy, new design, even a rebuilt machine learning model behind it. Nothing moved. Then someone finally asked an uncomfortable question: does anyone actually care about this part of the product?
They ran the test. Removing it was the biggest win the team saw all year.
This isn't a story about a bad team. It's the default pattern in most stalled optimization work, months of effort spent improving something nobody's actually confirmed matters. The fix isn't a better tweak. It's a sharper question, asked earlier.
This playbook hands you that question, and the judgment for when to use it, before your team burns another quarter on the wrong problem.
This playbook hands you that question, and the judgment for when to use it — before your team burns another quarter on the wrong problem.
What It Looks Like In The Room
Picture the next roadmap review. A PM is pitching Q3's big idea: full personalization, built on machine learning, three engineers, two quarters. Everyone's nodding. Then someone asks one question: "Have we tried just turning off the current version and seeing if anyone notices?"
The room goes quiet, not because the idea is bad, but because nobody's actually tested the assumption underneath it. That single question either saves two quarters of wasted build time, or it gets answered, and the team builds with real conviction instead of a hunch.
This playbook is what lets you be the person who asks that question, and back it up.
Why Download?
Get a single, repeatable technique for testing assumptions before you sink more time into optimizing the wrong thing.
✅ The technique itself — stated plainly, with the exact question to ask
✅ Why a negative test result is just as valuable as a positive one
✅ Two real examples where removal gave opposite, equally useful answers
✅ How to use this as a pushback tool before a team over-builds something complex
✅ When to push for full removal vs. when to break the idea into smaller steps instead
Who is this for?
- Growth & CRO Leads — You've watched a team defend the same underperforming page for two quarters, convinced it "just needs one more tweak." This gives you the question that ends the debate — and the evidence to redirect that effort somewhere it'll actually move the number you're accountable for.
- Product Managers — You're two weeks from greenlighting a roadmap item nobody's stress-tested. This is the low-drama way to ask "are we sure?" out loud, before three engineers and a quarter of runway get spent finding out the hard way.
- Analysts & Data Scientists — You're tired of "we think this matters" surviving another quarter on vibes. This turns that hunch into a decisive, evidence-backed answer in one test cycle, not another round of inconclusive dashboards.
- Experimentation Leads — Somewhere in your program right now, a test has been "optimizing" the same surface for months with nothing to show for it. This is the move that gets it unstuck — or confirms it's time to stop trying.
- Engineering & Design Leads — You're about to greenlight the complex build a stakeholder's excited about. This gives you a way to push back that reads as rigor, not resistance — before the sprint starts.