The Mirage of the Lower CAC

The first two posts in this series were about vocabulary and architecture: what “signal” means, and what’s actually happening underneath it.

This post is about the number that shows up at the top of nearly every case study in this category - some version of “CAC down X%” or “conversion up some multiple.” It’s worth asking what that number is actually standing in for, because with a targeting change specifically, it’s rarely just one thing.

CAC can be a stand-in for two different questions, not one. A lower cost to acquire a customer can mean the campaign got more efficient, same kind of customer, found more cheaply. Or it can mean the campaign found a different customer, one who happens to convert more easily, at a lower price, but isn’t necessarily worth the same amount once they’re acquired. Both produce the identical-looking headline. Only one of them is actually the win it’s presented as.

This distinction matters more for a targeting claim specifically than it does for most other marketing changes, and that’s worth pondering. If a brand improves its creative or its offer, it’s still talking to roughly the same pool of people; the change mostly affects how well the message lands within an existing audience. A targeting change is different in kind: it changes who’s in the pool at all. That’s the entire promise of “signal-composed audiences”: reach a different set of people than the old dropdown-category targeting would have found. Which means the quality question isn’t a minor caveat here, it’s close to the central question.

A signal-based system that’s optimizing toward a cheaper conversion event, rather than toward a valuable customer, will happily find you people who are easy to convert and worth less once they convert - and the CAC chart will look great the whole time.

The same ambiguity sits inside a “conversion up” number. Conversion of what, exactly, to what - a first purchase, an email signup, an add-to-cart? A dramatic lift in conversion on a shallow action is a different, much smaller claim than a lift in paying, retained customers, and the two get presented with the same enthusiasm.

The only way to actually tell these stories apart from the outside is to ask what happened after the acquisition moment the case study is measuring - repeat purchase rate, 90-day retention, actual lifetime value against the prior cohort. That data is almost never in the case study, for an understandable reason: it takes months to know, and a launch-week CAC number is available immediately. But its absence is exactly why a compelling CAC or conversion number, on its own, tells you less than it appears to. It’s a textbook example of Goodhart’s Law: when a metric becomes the target, it ceases to be a good metric. CAC is an efficiency metric being asked to also answer an effectiveness question, and most of the vocabulary in this category, like “signal,” “composed“ or optimized” doesn’t distinguish between the two.

Worth a quick, more basic caveat here too: even a legitimate efficiency gain can be an artifact of a weak comparison point - an untouched baseline, a flattering time window, a best-performing SKU standing in for a portfolio average. That’s a real issue, but it’s also a generic one that applies to any before-and-after marketing claim, signal-based or not.

The quality-of-customer question above is the one that’s specific to what audience-targeting claims to do, and it’s the one worth asking first. Is it likely that there is a large, untouched pool of high-quality customers that your existing tactics have been unable to reach?

There’s a deeper problem underneath all of this, though, one that goes beyond any single case study’s math, and it applies even to platforms with much better data than a typical before/after comparison - including the ones with real, verified purchase records instead of inferred behavioral signal. That’s where this series is headed next: exploring the difference between a platform that reports a number and one that’s actually tested it.

Next: why having real transaction data isn’t the same as having a validated number - and why even the platforms with the best data in the industry, including some with real purchase records, have been shown to overstate their own results.

Does this resonate? Start a discussion in the comments below.