Skip to content
Kuraib Ali
AI Search

How Does ChatGPT Actually Decide What to Cite?

Independent research reverse-engineered ChatGPT's retrieval system by capturing 1,200 real answers. What it found looks nothing like a search engine.

9 min readUpdated

Independent research published in August 2026 found that ChatGPT grounds its answers through three distinct layers: a discovery index that finds candidate pages, a reading cache that stores full copies of pages it has actually fetched, and a small set of pages it opens live during a conversation. Most users, the vast majority of whom are on ChatGPT's free tier, never trigger that third layer at all. Their answers come entirely from titles and short snippets sitting in an index most site owners have never heard of.

Three Layers, Not One Search Index

The research, conducted by RESONEO and published via Search Engine Land, captured and analyzed 1,200 ChatGPT answers, 88,000 search results, and 26,900 distinct pages by recording the raw data stream ChatGPT sends to the browser, including fields the interface itself never displays. That stream is not documented by OpenAI, which speaks publicly only of unnamed "third-party search providers," so everything here should be read as independent, reverse-engineered research rather than something OpenAI has confirmed.

The three layers behave completely differently from each other. A discovery index, internally identified in the data stream as labrador for OpenAI's own retrieval hub and bright or oxylabs for scraped Google results bought live, surfaces candidate URLs with a title and a short snippet, nothing more. A reading cache stores full page content, converted from HTML to Markdown, for any page ChatGPT has actually fetched at some point, shared across every user regardless of tier or location. And a live page open, the rarest event of the three, happens when the ChatGPT-User agent fetches a page in real time during a single conversation.

Why Most Users Never Get a Page Opened at All

The routing between these layers comes down to cost, not relevance. In instant mode, the fast, free default most ChatGPT users get, the researchers found zero pages opened in 93% of the answers they captured. The response leans entirely on titles and snippets already sitting in the labrador index. Opening a URL live, or paying for scraped Google results through bright, only happens in the slower, subscription-gated thinking mode, where a user has already agreed to wait.

That split matters commercially, since OpenAI has stated more than 90% of ChatGPT users are on the free tier. Free means instant, and instant means the internal index does almost all the work: on a free account, clicking "Think" pulled 74.7% of its results from labrador in the researchers' data, versus just 3.1% from classic scraped Google results, while paid thinking mode at the same effort level flipped that ratio to roughly 75% scraped Google and 25% labrador. Two people can ask the identical question and draw from almost entirely different underlying source pools depending on which tier they're on.

What a 200-Character Snippet Actually Contains

For the majority of ChatGPT conversations running on the internal index, the text a page is judged on is remarkably small. RESONEO's research found each result carries an untruncated title, however long, plus a snippet of roughly 200 characters anchored on the page's H1 heading. That snippet pulls in whatever visible text sits immediately around the heading, category labels, image alt text, a byline, a publication date, sometimes a table of contents, and in one case the researchers measured, a snippet that was entirely table-of-contents text with zero actual content. The meta description plays no role at all in this pathway; it's read only by the Google-fed pipelines, where it's used in roughly one of every three results.

The snippet is also frozen at indexing time. RESONEO found the same URL returns the identical snippet regardless of what the user actually asked, cut once and served as-is until the next crawl updates it. That's a noticeably cruder mechanism than query-dependent snippets, which conventional web search moved past years ago.

Angled sedimentary rock strata showing distinct layers of light and dark stone, representing the three separate layers researchers found in ChatGPT's retrieval system: a discovery index, a reading cache, and rare live page opens

The Reading Cache: Shared, Stale, and Not Watching for noindex

Separately from the discovery index, RESONEO documented a reading cache that keeps a full copy of every page ChatGPT has ever actually fetched, shared across every user and every tier. If one paying user in one country triggers a fetch of a page, a free user asking about the same page elsewhere gets served that same stored copy rather than a fresh one. The refresh pattern the researchers observed is a stale-while-revalidate model: a copy is treated as current for roughly 30 minutes, after which the next request still gets the stale version immediately while a background fetch quietly updates it for whoever asks next. Since the trigger for a refresh is user interest rather than a schedule, an unpopular page can sit unrefreshed indefinitely; the researchers documented copies still being served more than 90 days after the original fetch, with no eviction limit observed.

Two details about that caching layer are worth knowing on their own. The researchers found Cache-Control: no-store headers and noindex directives were both ignored by this pathway; neither stopped a page from being fetched and stored. And the Markdown conversion applied to cached pages strips scripts, iframes, and JSON-LD, meaning structured data added specifically for the reasons covered in structured data and AI citations doesn't reach the model through this particular path, even while CSS-hidden text was found to survive the conversion intact. Pages are also capped at exactly 4 MB; past that limit, RESONEO found the fetch rejected outright rather than truncated, so an oversized page isn't partially read, it's not read at all.

Rows of glass jars filled with preserved food stacked on wooden pantry shelves, representing ChatGPT's shared reading cache: full page copies stored and served to later users, some found still in use more than 90 days after the original fetch

The Analytics Blind Spot

OpenAI's own Publishers and Developers documentation confirms that ChatGPT appends the utm_source=chatgpt.com parameter to citation links shown to users, covered in more detail in how to track AI referral traffic in GA4. RESONEO's research adds a specific wrinkle to that picture: pages ChatGPT opens on its own during thinking mode carry no UTM parameter at all in the citations displayed afterward, since that tagging applies to the outbound link a user might click, not to ChatGPT's own internal fetch of the page. That means the pages a site owner would most want visibility into, the ones ChatGPT actually read in full, are exactly the ones a UTM-based tracking setup can't see. RESONEO's guidance is to also watch for the ChatGPT-User agent directly in server logs rather than relying on UTM parameters alone.

Being Retrieved vs. Being Cited: The Numbers That Matter

RESONEO's corpus makes the gap between showing up and actually mattering concrete. Across their capture, 61,332 URLs were surfaced into the sources shown to users, but only 5,032 became the lead source behind an actual citation, and just 759 pages were opened live, all of them in thinking mode. Of the pages ChatGPT actually opened and read, 74% ended up cited. Of the pages merely retrieved into the source list without being opened, only 7% were. Being present in the results is a weak signal on its own; being one of the rare pages ChatGPT actually reads is what correlates with getting named.

One brightly lit window in an otherwise dark apartment building at night, representing how rarely a retrieved page is actually the one ChatGPT opens and cites out of the many surfaced into its results

What This Research Suggests, Without Overreacting to It

RESONEO's own recommendations, grounded in what they measured rather than speculation, are modest: write titles as self-contained statements rather than truncation-optimized labels, since the full title reaches the model untruncated in the labrador pathway; put the actual substance of a page in the first 200 characters after the H1, since that's the only body text instant-mode answers see; keep meaningful alt text near the top of a page; keep the meta description anyway, since Google-fed pipelines still use it; and stay under the 4 MB fetch limit with content that doesn't depend on JavaScript execution, consistent with the crawler limitations covered in do AI crawlers actually read your site.

The researchers themselves caution against over-indexing on any single mechanism here. The internal field that first let them identify these pipelines disappeared from the data stream overnight partway through their study, and provider names and routing behavior shifted again within the same reporting window. A system this actively engineered by OpenAI can change the specific plumbing at any time, so the durable takeaway isn't the exact snippet length or cache duration, it's the underlying shape: a genuinely useful, clearly structured, easily crawlable page that answers a real question keeps paying off regardless of which internal pipeline happens to be serving a given answer this month.

What's Confirmed, What's Independent Research, and What's Still Unknown

Worth separating clearly, since this is one of the more mechanically detailed pictures of an AI system's internals published so far, but it isn't official documentation. Confirmed directly by OpenAI: that ChatGPT appends utm_source=chatgpt.com to citation links shown to users, and that a ChatGPT-User agent exists and can be identified in server logs. Independent research, not confirmed or denied by OpenAI: the specific pipeline names, the 200-character snippet mechanics, the 30-minute cache freshness window, the 4 MB fetch cap, and the retrieved-versus-cited percentages above. Genuinely unresolved even by the researchers' own account: which crawler actually populates the labrador index, since their own canary pages caught OAI-SearchBot nearly silent while that index kept returning results, and why snippet-less URLs get cited at a higher rate than URLs with snippets in instant mode, a pattern they documented but can't yet fully explain.

Related: do AI crawlers actually read your site, how to track AI referral traffic in GA4, structured data and AI citations, AEO, GEO, and SEO: what each term actually means.

Frequently asked questions

Has OpenAI confirmed how ChatGPT's retrieval system actually works?

No. The pipeline names and mechanics described here (labrador, bright, oxylabs, the reading cache) come from independent research by RESONEO, published via Search Engine Land in August 2026, based on captured data from real ChatGPT sessions. OpenAI has never documented these internal pipelines publicly and only refers generically to third-party search providers, so treat this as reverse-engineered research, not official confirmation.

Does my meta description matter for getting cited by ChatGPT?

It depends entirely on which retrieval pipeline serves a given answer. RESONEO's research found the meta description is ignored completely by labrador, OpenAI's own index, which powers nearly all of instant mode. It's used about one time in three by the Google-fed pipelines (bright and oxylabs), which mostly appear in the paid, slower thinking mode.

Why doesn't utm_source=chatgpt.com show up for every ChatGPT citation in my analytics?

Per this research, that UTM parameter is appended only to the clickable links displayed to users, the ones OpenAI's own documentation describes. Pages ChatGPT opens and reads on its own in thinking mode carry no UTM trace at all in the citations shown afterward, even though those opened pages get cited at a far higher rate than pages that are merely retrieved.

Does adding noindex or Cache-Control: no-store stop ChatGPT from caching my page?

According to this research, no. RESONEO documented ChatGPT's reading cache ignoring both signals, serving stored copies of pages more than 90 days after the original fetch with no observed eviction cap. This is independent testing, not a policy OpenAI has stated, but it's worth knowing if you were relying on either directive to keep a page out of ChatGPT's hands.

Part of the AI Search cluster.

Want this applied to your own site, not just read about it?

This is the free version, evidence-labeled and yours to read at no cost. Applying it to your own site (technical SEO, AI search visibility, and GEO in one pass) is separate, paid work at kuraib.site.