Research
What we measure about AI-search citation.
A roughly monthly dispatch, each a dated snapshot of one finding from our hand-run citation probes across ChatGPT, Perplexity, Claude, Copilot, and Gemini. The live per-term data lives on the Observatory; these pages are the narrative analysis. Follow via RSS or email.
Dispatches
- Dispatch #9 · A citation is not proof: three ways AI search cites you and still gets it wrongThe AI-search industry treats one question as the whole game: was I cited or not? But a valid citation has three independent properties: the source supports the claim, the citation points to the right source, and the version cited is current. Passing one says nothing about the other two. All three fail in the wild: an audit of AI-search verifiability finds about one in four citations does not support its sentence, and a Tow Center benchmark finds several engines emit fabricated or broken URLs in over half their citations; in our own probing an engine rendered our content under a domain we do not own, and another cited our live URL while reproducing figures we had already corrected. Our own two observations are rare; the point is that 'cited: yes' is not the same as citation integrity, which needs claim, destination, and version checks a binary counter never performs.
- Dispatch #8 · We built this glossary on coining our own terms. Does AI search actually reward it?A core selection rule of this glossary is to coin practitioner terms in open naming territory and to kill terms whose meaning is already contested. We tested that rule against fourteen rounds of our own AI-citation probes. Empty-coined terms were our highest-cited territory (38 percent) and contested-bare ones the lowest (4 percent), but the effect is confounded with editorial effort: the cleaner comparison, established at 15 percent versus contested-bare at 4 percent, is closer to fourfold. The association is also highly per-engine (a large coined edge on Gemini and Claude, roughly none on ChatGPT), and Gemini's edge is substantially Google rank in disguise. Within the coined terms, competitor count did not predict citation breadth, which shows monopoly is not what protects the citation, though it cannot by itself prove what does.
- Dispatch #7 · We refuse to invent benchmarks. Does AI search punish us for it?Several of our glossary entries decline to give the number people search for: no target citation match rate, no standard attribution rate, no proven lift from authoritative tone. We tested whether that honesty costs us citations by asking five AI engines the number-seeking version of each question and comparing against the definitional control. In the expanded round, 16 of 25 pairs were eligible because the definitional prompt cited us; the number-seeking variant kept the citation in 11, and no loss went cleanly to a page that asserts a number. On the engines that carry editorial framing, the answers adopted our hedges and debunked the inflated figures. The honest reading is conditional: honesty held on the engines that transmit framing, where we hold the term.
- Dispatch #5 · GEO's most-cited numbers, checked against the papers they come fromThe GEO field rests on a few foundational studies, but the headline figures travel into the marketing with the methodology stripped off. We read the three papers behind the numbers against how they are cited. The famous '40% boost' is a position-adjusted word-share proxy on a 2023 GPT-3.5 testbed, and the paper's own real-engine headline quietly switches to a different metric; a multi-actor re-test found most content tactics ineffective or negative; and a verifiability audit shows only about half of AI-generated sentences are fully supported by their own citations.
- Dispatch #4 · One panel, five engines, mostly separate citation sets: 'cited by AI' is not one thingWe ran the same prompt for the same pages against five AI engines in one round. Of the eighteen pages any engine cited, ten were cited by a single engine and none by more than three. A citation on one engine barely tells you about the next. In this panel, each engine behaved like its own citation surface.
- Dispatch #3 · Cited more on Gemini, less on ChatGPT: a gradient, and what it is notOur pages are cited as AI sources far more on Gemini than on ChatGPT, and most of what Gemini cites us for is vocabulary we coined. The flattering read is that Gemini prefers our original work. We ran the test that separates that from Google-rank mirroring, and the flattering half did not survive.
- Dispatch #2 · Google caught up: the AI-citation gap looks like a reporting lagLast month, AI engines cited a page Google's index report showed as unindexed. We said we would track whether Google caught up. It did, and the best-supported reading is a reporting lag, with Gemini as the clock that nearly proves it.
- Dispatch #1 · AI engines cited this page before Google indexed itThree AI engines cited our citation-precision page while Google Search Console still showed it unindexed. An honest look at the co-occurrence, with caveats.