The page that read better sold fewer books.

What we ran

A page rebuild tested against its own control. Two small sessions, 79 on the control and 75 on the rebuild, both underpowered for a real statistical read.

What the numbers showed

48% of visitors on the rebuild got past the hero section. The control managed 31.6%, a gap of about 16 points in the rebuild's favour on the reading metric. On sales, the control took three purchases. The rebuild took one.

Why it failed

The version that won on reading lost on money. Reading further down a page is not the same thing as buying, and a page can be rewritten to hold attention longer without rewriting the one thing that actually moves a reader to the button. At this sample size, a significance test would have told us little either way. A plain before-and-after comparison on settled sales was the more honest read of what actually happened.

What we changed

Settled cash decides a page test, not an engagement metric. Scroll depth, time on page and read-through all still get tracked, but none of them get to overrule the purchase count. If a rebuild reads better and sells worse, the rebuild loses, full stop.

How to spot it in your own account

If you are running a page test and judging the winner on scroll depth, time on page, or any engagement score, stop and pull the purchase count for the same two sessions instead. A page that reads better is a page that is easier to finish, not a page proven to sell more. Run the comparison on sales before you declare a winner, even at small sample sizes where a formal test cannot settle it. And check your session count against the sample-size arithmetic before trusting either result: at a 2% baseline purchase rate, detecting a real 50% lift needs roughly 3,136 sessions per arm. 79 and 75 sessions were nowhere near that.