Agentic AI

The Model Is Not the Market: The Limits of Synthetic Respondents

9 min read
A single figure whose shadow fans into a crowd of identical silhouettes, synthetic respondents as one model projecting many.

Picture the choice a consumer-research team faces this year. On one side, a real panel: weeks of fieldwork, incentive costs, recruiting, the slow grind of getting a few thousand actual people to answer honestly. On the other, a synthetic panel: a language model that returns a thousand "consumer" reactions before you finish your coffee, at a fraction of the cost, on any question you can phrase.

Reach for the second one. It is a reasonable instinct, and I want to be fair to it. I have spent enough time building on top of these models to know how seductive the fluency is. Ask a good model what a 34-year-old parent in Pune thinks of your new snack brand and it will answer, confidently, in plausible prose, every time. It never fatigues, never no-shows, never asks for a gift card.

That is exactly the thing to be careful about.

The pitch is not fringe anymore. Qualtrics launched AI synthetic respondents this spring with claims of large accuracy gains over general-purpose models. YouGov, Toluna, and a wave of startups are selling variations of the same idea, and at least one industry chief has called synthetic data the single biggest shift in research. Buyers are already purchasing "insight" generated without a single human in the loop.

So it is worth saying plainly what a synthetic respondent is, and what it is not.

The model is not the market.

The reasonable-looking trap

The trap is treating a model's fluent, plausible reaction as a sample drawn from a real population. It looks reasonable because in aggregate the output often fits. Run the synthetic panel, plot the distribution, and it lands close enough to real data in a demo to feel real.

Then you look at what "close enough" is hiding.

A benchmark released this year, ConsumerSimBench, took 1,553 real topics and more than 23,000 rule-audited criteria of how real people actually reacted, and asked frontier models to reconstruct those reactions. The best model covered 47.8% of the criteria. Under half. Several models with strong scores on conventional technical benchmarks did worse, not better. The competence that makes these systems good at reasoning tasks did not transfer into faithfully reproducing what a population actually thinks.

A related line of work points the opposite way from what you would expect. In experiments that put language-model agents into market-style decisions, the agents tend to look too rational, matching real human behavior only in part. They lean toward the tidy, optimizing choice and underproduce the messier, anomalous behavior that is often the whole reason you were studying people in the first place.

Put those together and the shape of the problem appears. The synthetic panel is confident everywhere and accurate mostly in the boring middle. It has language competence. It does not have a grounded population.

The edges are where the money is

The machine-learning literature already named this failure mode, in a different context, a while ago. Train a model recursively on its own generated output and the distribution collapses inward: the rare cases, the tails, disappear first, and everything drifts toward the mean. Researchers studying synthetic samples in social research describe the same effect in plainer terms, that models flatten variance and cluster answers around the average.

Consumer insight lives in the tails. The regional deviation that becomes a national trend. The early adopter who signals where the category is going. The one segment quietly abandoning you while the aggregate still looks healthy. A measurement instrument tuned to the fat middle is blind in precisely the place the decision value sits.

Synthetic respondents are confident in the middle of the distribution and silent at its edges. The edges are where the money is.

There is a second, quieter distortion. Because you can generate an arbitrarily large synthetic sample, tiny differences can look statistically significant even though the draws are not independent observations. That does not measure certainty. It manufactures it. And models carry a bias toward the answer the prompt seems to want, the same sycophancy problem that turns a helpful assistant into a flatterer. I have written before about what happens when an advice-giving model optimizes for agreement instead of accuracy; a research instrument that drifts toward the result you were hoping to find is not measuring anything. It is agreeing with you at scale.

The loop that cannot close

Here is the structural core, and it is where a systems lens earns its keep.

The honest versions of the pitch validate their output against real human data. That is the honest part, and it is also the undoing, because the validation requires the very thing the product is meant to replace.

I evaluate agent architectures with a framework I call SRAL: State, Reason, Act, Learn. A synthetic panel is a powerful Reason engine sitting on top of no faithful State, with no Learn loop to correct it. It interpolates plausible reactions from what it already absorbed. It has no channel back to what real people actually did this quarter.

Anyone who has run a large system knows the rule this breaks. You never trust a cache you cannot reconcile against a source of truth. What the industry is doing right now is standing up a fast, read-heavy cache, the synthetic panel, and quietly retiring the origin, the real fieldwork, then validating the cache against a snapshot of an origin that goes stale the moment consumer behavior moves.

A cache you can never reconcile against the source is not a cache. It is a guess with good latency.

The circularity is the part that should keep you up at night. The more the industry substitutes synthetic for real to save cost and time, the more it thins the ground-truth corpus that made synthetic trustworthy in the first place. The validation set and the thing being replaced are the same asset. Spend it in one place and it is gone from the other.

Ground truth is thinning from both ends

The real side is no longer the stable anchor it used to be either. In one compensated online health study this year, fewer than one in ten eligible screener entries could be verified as legitimate and unique. The rest were duplicates, bots, or professional survey-takers. It is a single study in a single field, and it should be read that way, but it points at a quality problem the industry has been naming for a while: the data flowing into panels is not automatically clean just because it is nominally human.

The symmetry is uncomfortable. Synthetic panels are openly artificial and unvalidated at the level of individual decisions. Real panels are nominally human and increasingly contaminated. Both roads arrive at the same place: it is getting genuinely hard to know whether a given consumer signal came from a real person making a real choice.

This is the same tension I traced from the platform side in quick commerce, where the party running the transaction also owns the scoreboard, and from the buyer side in agentic commerce, where the shopper's consideration moves onto surfaces the retailer cannot see. Different mechanism, same fracture. The measurement is drifting away from the thing it claims to measure, and nobody has reconciled that. The industry is scaling a technology whose trustworthiness rests on a foundation it is simultaneously eroding.

That is the open problem. I am going to leave it there, as tension, because it is genuinely unresolved.

What this is not

Precision matters, so let me scope the critique.

Synthetic respondents are not useless, and the claim here is not that they are a trick. As a Reason engine over existing knowledge they are genuinely useful: pretesting a questionnaire before you field it, stress-testing hypotheses cheaply, generating directional ideas, giving coverage in places where no data exists at all. Used that way, they accelerate the work.

The failure is narrower and more specific. It is substituting synthetic output for ground-truth, decision-grade measurement of what real people actually do, and then treating that output as if it carried the authority of observation. The technique is fine. The substitution is the mistake.

It does not help that the profession has not agreed on what synthetic data even is. Industry bodies are still drafting definitions and disclosure norms, and those norms are voluntary and lagging the product by a wide margin. Buyers are purchasing "insight" without a standard for whether it came from a human at all.

The signal you removed

The deepest problem is the one SRAL points at directly. A Learn loop breaks when its ground truth disappears. A system that trains on its own priors and checks itself against a fading snapshot of reality cannot detect its own drift. It grows more confident and less connected to the world at the same time, and nothing inside the loop will warn you, because the warning would have to come from the signal you stopped collecting.

I have argued before that learning is not storing what happened, it is distilling what to do differently, and that the distillation is only as good as the verifiable signal underneath it. Synthetic panels are what you get when you keep the distillation and quietly remove the signal.

The model can tell you what a plausible consumer might say. It cannot tell you what the market did. Those are different questions, and only one of them can be checked.

The model is not the market.

Sources

The figures here are reported by the vendors and papers cited, not independently verified. Read them as directional.