June 5, 2026 · The Caliper team

How to ask “how did you hear about us” without ruining the answer

The question is one line. “How did you hear about us?” Anyone can ship it in an afternoon. That is exactly the problem. Because it is so easy to put on screen, teams treat it as solved the moment it renders, and then they make decisions off data that was corrupted by the form itself before a single user lied to them.

The failures are not bad luck. Each one is a mechanical consequence of a specific design choice. Walk through them in the order they bite, because the fix for each is just as mechanical.

Failure mode: the option list never moves

Show the same option list in the same order to every user and you will over-index whatever sits at the top. If your options are alphabetical — “App Store,” “Facebook,” “Friend,” “Google” — “App Store” collects clicks it did not earn, partly from people who genuinely don’t remember and grab the first plausible thing, partly from satisficing. The order of the list becomes a thumb on the scale, and because it’s the same thumb every time, the bias is systematic. It does not average out. It compounds into a channel mix that quietly rewards whatever you happened to list first.

The fix: randomize option order per impression. Not per session, not per user — per impression. When the order is different for everyone, position bias scatters into noise instead of concentrating on one option, and the signal you actually want survives. This is the cheapest high-value change available and the one most homegrown surveys skip.

Failure mode: the open text field

A free-text box feels like the honest choice. It is the opposite. Ask users to type how they heard about you and you get “tiktok,” “TikTok,” “tik tok,” “saw a video,” “ig,” “instagram reel,” and a thousand other spellings of four or five real channels. Now someone has to classify that mush. Either a human does it by hand and you’ve built a part-time job, or a script does it with a regex that silently miscategorizes the long tail. Open text does not capture more truth; it relocates the truth into a column nobody can aggregate without re-introducing exactly the judgment calls a structured question would have made up front — but worse, after the fact, with no consistency.

The fix: closed options drawn from a maintained taxonomy. Closed does not mean rigid — it means the answer space is a controlled vocabulary you curate, so “TikTok” is one value and not eleven. The taxonomy is the asset. It has to be maintained, because channels appear and die, but a maintained list of forty clean options beats an unbounded field of dirty strings every single time you try to compute a number from it.

Failure mode: the influencer answer with nowhere to go

Closed options have one hard edge: creators and shows. You cannot list every podcast and every influencer as a checkbox — there are thousands, and the list would be useless. So teams either omit them or bury them under a generic “podcast” or “influencer” bucket. Either way the specific creator who drove the install is lost, which is brutal, because creator-driven channels are precisely the dark ones your other tools already can’t see. You finally have a chance to learn that a specific host moved installs, and the form throws it away.

The fix: auto-suggest layering for creators and shows. The user picks a category — “podcast,” “a creator I follow” — and a type-ahead field resolves their input against a known list of shows and creators. They start typing, they select the real entity, and you capture the specific source instead of the bucket. This is the one place a typed input belongs, because it is constrained by suggestion rather than open to the wind.

Failure mode: the tiny skip button

If the survey is mandatory, or the skip control is a five-pixel “x” in a corner, you are not collecting answers. You are collecting guesses. A user who genuinely does not remember, faced with a wall they cannot pass, will tap something to make the screen go away. That tap looks identical to a real answer in your data, and it is worse than no answer, because it is noise wearing the costume of signal. You will trust it. You shouldn’t.

The fix: make skip prominent. A skipped survey is honest missing data you can account for; a guessed answer is corrupted data you can’t distinguish from the truth. Optimizing for completion rate over answer quality is the most common way teams ruin this question, and it comes from the instinct that more rows is more data. It isn’t.

Failure mode: pretending the responders are everyone

Even with perfect form design, not everyone answers, and the people who answer are not a random sample. The memorable touches over-respond. If you take your raw response percentages and multiply them against installs, you’ve assumed the silent majority looks exactly like the vocal minority, which they don’t.

The fix: model the non-responders. Build response-rate-aware estimates that treat non-response as its own population rather than as more of the same. This is the difference between a survey result you can put next to your MMP and SKAN data and one that just makes a nice pie chart.

Failure mode: asking at the wrong moment

Timing decides answer quality before content does. Fire the survey mid-onboarding and you’ve inserted friction between a user and the thing they came to do; they will skip, guess, or churn, and the answers you get are contaminated by the interruption. Fire it before the user has committed to anything and you’re asking someone who hasn’t decided to stay.

The fix: fire once, after first open and after the conversion moment, when the user is past the friction and the memory of how they got here is still fresh. Once. Re-prompting trains users to dismiss reflexively and poisons the well for every future question.

The shape of a question that survives

Put the fixes together and the question stops being a one-line afterthought: closed options from a maintained taxonomy, randomized per impression, auto-suggest layering so creators and shows resolve to real entities, a prominent skip so missing data stays honest, fired once after the conversion, and non-responders modeled rather than ignored. None of it is exotic. It is just the set of decisions that determine whether the answer is worth joining to anything.

Caliper is built around this shape rather than the one-line version — the taxonomy, the randomization, the skip, and the non-responder modeling are the product, not the text on screen. The hard part of this question was never asking it. It was not ruining the answer.