.svg)
Guardrail Metrics Explained: What They Are and How to Catch Cannibalization
It's the experiment result every team celebrates and later regrets: the primary metric went up, you shipped, and a month later something else — retention, revenue, page speed — had quietly gotten worse. The win was real; so was the hidden cost. Guardrail metrics exist to catch that hidden cost before you ship, and they're the single best defense against one experiment's win coming at another's expense.
The short version: A guardrail metric is one you track during an experiment not to improve, but to protect — it shouldn't get worse, even though it isn't what you're trying to move. Guardrails catch the hidden costs of a change: a variant can lift your primary metric while degrading page speed, increasing errors, hurting retention, or stealing conversions from another team. Their most powerful use is catching cannibalization — by including other teams' primary metrics as your guardrails, you'll see when your "win" comes at their expense. The rule that puts them to work: ship only if your primary metric improves and no guardrail goes negative.
What is a guardrail metric?
A guardrail metric is a metric you monitor in an experiment to make sure a change doesn't do harm you didn't intend. You're not trying to move it — you're making sure it doesn't move the wrong way. Where your primary metric answers "did this work?", guardrails answer "did this break anything?"
The distinction matters because most changes have side effects. A more aggressive email cadence can lift click-throughs while driving unsubscribes. A flashier page can lift engagement while slowing load times. Guardrails are how you see the whole picture instead of just the number you hoped would move.
Where guardrails fit: the four metric buckets
Guardrails are one part of a complete metric structure. The cleanest way to organize an experiment's metrics is into four buckets: a primary metric (the single decision-maker), secondary metrics (behavioral signals close to the change), guardrail metrics (what must not get worse), and learning metrics (exploratory signals). We break down the full structure in The 4 Metric Buckets Every Experiment Needs — this resource zooms in on the guardrail bucket, because it's the one teams most often skip, and the one that quietly protects the business.
Common guardrail metrics
Guardrails vary by what you're testing, but the usual suspects fall into a few groups:
- Performance and reliability: page-load time, error rate, crash rate. Almost any change could slow or break something, so these are near-universal guardrails.
- User experience and satisfaction: unsubscribe/opt-out rate, support-ticket volume, bounce rate. A change that annoys users often shows up here first.
- Retention and long-term value: retention, repeat usage, revenue per user. A short-term win that costs you long-term engagement is a bad trade — and only a retention guardrail will catch it.
- Other teams' primary metrics: the guardrails that catch cannibalization (more on this next).
How guardrails catch cannibalization
This is the highest-leverage use of guardrails, and the one most teams miss. When you run experiments across multiple product areas, a change that lifts your numbers can quietly steal from another team's. Your dashboard looks like a win; the company's net position is flat.
The fix is simple to state: put the other teams' primary metrics into your guardrails. Now, if your homepage test lifts sign-ups but tanks the pricing team's conversion, the guardrail flags it before you ship. The change becomes a rule anyone can apply: ship only if the primary is positive and no guardrail goes negative. That single line turns a messy debate over conflicting metrics into an unambiguous decision — and it's why cannibalization stops being an invisible tax on your program.
How to choose your guardrails
You don't need dozens. For any experiment, ask two questions: what could this change plausibly harm? and what must never regress, no matter what? The answers are your guardrails. Keep the set small and stable — a handful that everyone agrees on beats a sprawling list nobody checks. And align across teams: if everyone protects the same shared guardrails, your whole program pulls in one direction instead of quietly working against itself.
One nuance on reading them: a guardrail doesn't have to be perfectly flat to pass. The practical bar is "no meaningful negative movement" — a tiny, non-significant wobble isn't a reason to kill a genuine win. Decide up front how much movement counts as a real regression, so guardrails inform the decision rather than blocking every test.
Why this protects your program's credibility
Experiments without guardrails produce wins that look great and don't hold up — the kind that ship, then mysteriously fail to move the annual number. That's how programs lose trust. Guardrails are part of the discipline that keeps results honest, right alongside not calling wins too early and following solid A/B testing best practices. A program that can prove it didn't break anything is a program leadership will keep backing.
Put the full framework to work
Guardrails are one bucket of four, and they work best as part of a complete metric structure. We've packaged the whole system — primary, secondary, guardrail, and learning metrics, with how to choose and score each — into a free download.
Get The Metric Bucket Framework →
If you want help building this discipline into how your team actually runs experiments, our Experimentation Programs are built to set up the metric structure, guardrails, and cadence that make results trustworthy.
Frequently asked questions
What is a guardrail metric?
A guardrail metric is one you monitor during an experiment to make sure a change doesn't cause unintended harm. You're not trying to improve it — you're ensuring it doesn't get worse. Examples include page-load time, error rate, unsubscribe rate, and retention.
What's the difference between a guardrail metric and a primary metric?
The primary metric is the one that decides whether you ship — the result you're trying to improve. A guardrail metric is one that must not get worse as a side effect. A test succeeds when the primary improves and no guardrail regresses.
What are examples of guardrail metrics?
Common ones include performance and reliability metrics (page-load time, error rate, crash rate), experience metrics (unsubscribe rate, support tickets, bounce rate), long-term metrics (retention, revenue per user), and — critically for catching cannibalization — other teams' primary metrics.
How do guardrail metrics prevent cannibalization?
By including other product areas' primary metrics as your guardrails. If your change lifts your own metric but hurts another team's, the guardrail surfaces it before you ship — so a "win" that merely steals from elsewhere gets caught instead of celebrated.
How many guardrail metrics should an experiment have?
Only as many as needed — usually a small, stable set everyone agrees on. Ask what the change could plausibly harm and what must never regress; those are your guardrails. A short list that gets checked beats a long one that gets ignored.
