THE MEASUREMENT CHEAT SHEET
Measurement is how you know whether your marketing worked. Every team I have worked with measures too much and understands too little. But the most expensive mistake is not measuring too much, it is confusing the metric with the goal. So before anything else: know your goal, and be clear that a metric is not your goal. The rest of this sheet only works once that is settled.
Not to Be Confused With
🎯 Attribution
Attribution divides credit between touchpoints. Measurement asks whether anything changed at all. A channel can win every attribution argument and still add nothing incremental. That is why attribution is for steering and incrementality is for truth.
📊 Dashboards
Telemetry tells you what happened. Measurement tells you whether it mattered and what to do next. A dashboard nobody acts on is wallpaper, not measurement.
🧮 Media mix modelling
Models estimate what happened; experiments prove it. I love experiments, but I hate modelled results on their own. Use models only after running experiments, and back-test them periodically.
P0: Know your goal
Write your goal down in one sentence. Not a number, an outcome. “Grow the business profitably.” “Become the default tool for X.” The goal is the thing you actually want; the metric is just the instrument you watch to check you are getting there.
This sounds obvious and almost nobody does it. Teams inherit dashboards, copy whatever their last company measured, and slowly the metric becomes the goal. That is Goodhart’s law: when a measure becomes a target, it ceases to be a good measure. People optimise the number instead of the outcome — more signups that never activate, more clicks that never convert.
So: goal first, in words. Then choose metrics that serve it. If you cannot state your goal without pointing at a dashboard, you do not have a goal — you have wallpaper.
The Metric Hierarchy
1) North-Star Metric
The single number that captures the value customers get from your product.
Use when
- Everyone can explain how their work moves it
- It reflects customer value, not company vanity
Avoid when
- It becomes a target people game (Goodhart's law bites fast)
- It is so lagging that nobody can act on it
2) Input Metrics
The levers that move the north star: the things teams can actually act on.
Use when
- Each input has a clear owner with real leverage
- You can draw the causal line to the north star
Avoid when
- You track twenty of them — pick the few that matter
- Inputs are chosen because they are easy, not because they matter
3) Guardrail Metrics
Things that must not get worse while you optimise everything else.
Use when
- You are pushing hard on growth — that is when damage happens
- You set explicit thresholds, not vibes
Avoid when
- Never — everyone needs at least one guardrail
4) Vanity Metrics
Numbers that go up and to the right and change nothing.
Use when
- You need a headline for a slide
- Honestly, that is about it
Avoid when
- Making any real decision
- Evaluating a team or a campaign
Rules of Good Measurement
- Start from the decision, not the data. Write the decision down first, then ask what evidence would change your mind. Data first is how you end up “discovering” whatever flatters you.
- Define the metric before you look at the numbers. Peeking first is how definitions quietly drift toward whatever looks good this quarter.
- Distrust ratios on their own. A conversion rate can rise because the numerator grew or because the denominator shrank. Always check both before celebrating. And remember what it takes to detect a real move: you can need a holdout half the size of the campaign just to show a 10% lift at 95% confidence. Treat small wiggles as noise until proven otherwise. Ratios are gameable too. Active users can be driven by clicks from users who bounce after one second; revenue by fraud users whose chargebacks land sixty days later.
- Segment before you celebrate. An average can improve while every underlying segment declines. (Simpson's paradox: that one is not from the book, it is from my own practice.) Cut by channel, cohort, and region, and when you compare markets match the trend as well as the level. Two states can both show 20% of people using WhatsApp while one grows 20% a month and the other 1%: read the first against the second and you will conclude nothing.
- Compare against a counterfactual. “Sales went up 10%” means nothing until you ask what would have happened without the campaign. (This is where incrementality comes in.) The workhorse is matched markets: run in one market, hold out a similar one, read the difference. My first ever incrementality test was eBay UK's first TV ad, measured this way when I was a junior analyst. I still use it to back-test randomised results, switching paid search off in one country while the rest of the world acts as the control. Match on your North Star metric, similar levels and similar trends, and check the channel mix: India and Pakistan look similar until you remember TikTok was banned in one and available in the other.
- Know when to stop measuring. I have seen teams spend more time debating attribution windows than the campaigns were worth. Perfect measurement has a price. Turning paid search fully off for a month in every country would cost about 10% of annual impact, so we test a few countries a year and model the rest. Measure enough to decide well, then go back to doing the work.
The test
A metric that cannot change a decision is decoration. Delete it. If your dashboard lost half its charts tomorrow, the half nobody misses was never measurement. It was wallpaper.
Know Its Limits
Not everything that matters can be measured, and not everything that can be measured matters. Brand affinity, creative quality, the trust a customer feels: all real, all hard to put a number on. If your measurement system has no room for things it cannot count, it will slowly optimise them away.
And don’t form a tyranny of measurement. It is a tool. I have seen teams spend more time debating attribution windows than the campaigns were worth, and the debate felt rigorous while the business learned nothing. When the metric and the anecdote don’t match, generally the anecdote is true.
My position: I don’t trust a team that never misses its goals. Either they are measuring wrong or the goals are far too easy. A measurement culture with a 100% hit rate isn’t a measurement culture. It’s theatre.