“People don’t remember how they heard about you” is a testable claim
There’s a cohort of marketers who’ll tell you survey attribution doesn’t work because people don’t remember where they heard about a product, or won’t admit it was an ad. You’ll find this take in every thread about measurement, stated with complete confidence and, almost always, zero data.
Which is ironic, because it’s one of the easier claims in marketing to test.
The spend-share test
If survey answers were mostly noise — bad memory, social desirability, random tapping — they would not track your spend. There’s no mechanism by which thousands of people independently misremembering would happen to misremember in proportion to your media mix, and then shift their misremembering when the mix changes.
So: take your channel spend shares over a period. Take your survey response shares over the same period. Compare. If the survey is noise, the two have no relationship. If people broadly remember, the lines track — not perfectly, and the gaps are themselves informative, but unmistakably.
Fairing (Caliper’s parent) runs this question at checkout for ecommerce brands, and where we’ve had access to a brand’s spend data, this is exactly the comparison we run. The shares track. When budget moves into a channel, the survey mix moves with it. People who just bought something can, in fact, tell you how they found you — and the “won’t admit it was an ad” theory has a particular problem, because paid channels show up in survey data at shares consistent with their spend.
The gaps are the product
The interesting part isn’t that the lines track. It’s where they don’t.
One brand we work with was running significant podcast and creator budget that barely showed up in their survey data — which looked, at first, like evidence for the skeptics. It wasn’t a memory problem. It was a form problem: the responses were landing in “Other” because the survey didn’t give creator-driven discovery anywhere structured to go. Recategorizing those responses and adding type-ahead resolution for shows and creators improved their measured podcast CPA by 9.2% — conversions the channel had earned all along, finally booked to it.
That’s the general shape of survey-skeptic evidence, in our experience: a badly built form producing bad data, read as proof that asking doesn’t work. Fixed option order, no escape hatch for “I don’t remember,” creator channels buried in a generic bucket — each of these failure modes manufactures the noise the skeptics then point at.
Memory has error bars. So does everything else.
None of this means recall is perfect. It isn’t — people compress “saw it on TikTok three times, then a friend mentioned it” into one answer, and the memorable channels over-respond. That’s why you model non-response instead of multiplying raw percentages, and why the survey is one leg of triangulation rather than a replacement for your MMP.
But hold the comparison honestly. Last-click books every podcast install as organic — a 100% error on those channels, presented in a dashboard with no error bars at all. SKAN nulls small campaigns entirely. Against that, an answer from the actual human, minutes after install, with known and modelable bias, is not the soft data point in the stack. It’s frequently the hardest one you have.
“People don’t remember” is a hypothesis. Run the spend-share test on your own data and find out. If your survey shares don’t track your spend at all, don’t conclude that memory is broken — check the form first.
Caliper is in early access — we're onboarding our first cohort.
Get access