By Ryan Richardson · Published 8 October 2026
Diagnosing properly when a number moves is step 58 (Part IX), marked hard difficulty, almost nobody does this. Most people get diagnosis wrong by skipping the procedure entirely, not by lacking information.
Instrument first: a deployment, a tracking change, or an expired credential. Then the chain in fixed order: traffic composition, page behaviour, checkout, order value, delivery. Stop at the first link that moved. When everything downstream looks broken at once, the likely cause is the instrument, not the business, and the first check is any recent deployment or tracking change. When spend flatlines or drops to zero, the likely cause is billing, not creative; check for a declined card or a paused payment method first. When revenue drops but traffic looks normal, the problem sits further down the chain: page, then checkout, then order value, then delivery, in that order.
Whether something happened at all needs almost no data. Whether a rate has genuinely changed needs real numbers, usually dozens of conversions. Waiting for a big-enough sample to answer the first, easier question is expensive, and more common than it should be. With zero events in n sessions, the 95% upper bound on the true rate is about 3 divided by n, which is often enough on its own to stop a bad idea without waiting for a larger sample.
Size the gap in dollars, name three or more candidate causes with the share of the gap each could explain, and rule out by number rather than by which explanation sounds most plausible. Check the change log before theorising: more entries in it than there have been weeks is itself the finding. Change one thing at a time; changing four and watching the number recover teaches nothing, because next time all four get changed again and it's still not clear which one mattered.
Naming a single cause (creative fatigue, an algorithm change) before checking the instrument, which is the satisfying explanation and usually the wrong one. Chasing a revenue drop that was a broken tag, which can cost a fortnight optimising a funnel that was fine the whole time.
| Claim | Value | Source |
|---|---|---|
| Fixed diagnostic order | instrument first (deployment, tracking change, expired credential), then traffic, page, checkout, order value, delivery, in that order | Measured in Real Money, Field Manual |
| Rule of three for zero-event samples | with zero events in n sessions, the 95% upper bound on the true rate is about 3/n | Measured in Real Money, Field Manual |
| Sample size needed to confirm a rate change | usually dozens of conversions | The Sixty Steps manuscript |
| Matrix status for this step | Hard difficulty, almost nobody does this, automated tooling | The Sixty Steps matrix |