A bigger raw purchase count can still be the worse audience

The symptom

Two targeting approaches are compared by purchase count, and one looks like a clear winner. The decision to favour it is made before checking what each side actually spent to get there.

The cause

A raw purchase count is a function of spend as much as it's a function of audience quality. Across ten ad sets and ten purchases, broad targeting beat a hand-built interest audience by cost per purchase, roughly two to one. But by raw purchase count alone, broad led nine to one, which is the number that's tempting to write down. It's the wrong one: broad had over four times the spend behind it, and the interest side's entire result rested on a single purchase.

The fix

Judge every targeting comparison on cost per purchase, never on raw purchase count, and check the spend behind each side before trusting either number. A comparison resting on a handful of purchases, especially one side resting on a single purchase, is too small a sample to generalise from, whatever the ratio looks like.

How to spot it

Before declaring a targeting approach the winner, pull the spend figure behind each side's purchase count and recompute cost per purchase directly. If one side's result rests on one or two purchases, treat the comparison as provisional rather than decided, and let it run longer before acting on it.