Skip to content
Kuraib Ali

How ChatGPT decides what to cite

Independent research reverse-engineered ChatGPT's retrieval system by capturing 1,249 real answers. What it found looks nothing like a search engine.

By Kuraib AliPublished Updated 9 min read

Angled sedimentary rock strata showing distinct layers of light and dark stone, representing the three separate layers researchers found in ChatGPT's retrieval system: a discovery index, a reading cache, and rare live page opens
Photo: Unsplash

Independent research published in July 2026 found that ChatGPT grounds its answers through three distinct layers: a discovery index that finds candidate pages, a reading cache that stores full copies of pages it has fetched, and a small set of pages it opens live during a conversation. Most users, the vast majority of whom are on ChatGPT's free tier, never trigger that third layer at all. Their answers come entirely from titles and short snippets sitting in an index most site owners have never heard of.

Three layers, not one search index

The research, conducted and published by RESONEO, captured and analysed a corpus of 1,249 ChatGPT answers by recording the raw data stream ChatGPT sends to the browser, including fields the interface itself never displays. That stream is not documented by OpenAI, which speaks publicly only of unnamed "third-party search providers," so everything here should be read as independent, reverse-engineered research rather than something OpenAI has confirmed.

The three layers behave completely differently from each other. A discovery index, internally identified in the data stream as labrador for OpenAI's own retrieval hub and bright or oxylabs for scraped Google results bought live, surfaces candidate URLs with a title and a short snippet, nothing more. A reading cache stores full page content, converted from HTML to Markdown, for any page ChatGPT has fetched at some point, shared across every user regardless of tier or location. And a live page open, the rarest event of the three, happens when the ChatGPT-User agent fetches a page in real time during a single conversation.

Why most users never get a page opened

The routing between these layers comes down to cost, not relevance. In instant mode, the fast, free default most ChatGPT users get, the researchers found ChatGPT essentially never opened a page. The response leans entirely on titles and snippets already sitting in the labrador index. Opening a URL live, or paying for scraped Google results through bright, only happens in the slower, subscription-gated thinking mode, where a user has already agreed to wait.

That split matters commercially, since OpenAI has stated more than 90% of ChatGPT users are on the free tier. Free means instant, and instant means the internal index does almost all the work. On a free account, clicking "Think" pulled 74.7% of its results from labrador in the researchers' data, against 3.1% from scraped Google results. Paid thinking mode at the same effort level flipped that ratio to roughly 75% scraped Google and 25% labrador. Two people can ask the identical question and draw from almost entirely different underlying source pools depending on which tier they're on.

What a 200-character snippet contains

For the majority of ChatGPT conversations running on the internal index, the text a page is judged on is remarkably small. RESONEO's research found each result carries an untruncated title, however long, plus a snippet of roughly 200 characters anchored on the page's H1 heading. That snippet pulls in whatever visible text sits around the heading: category labels, image alt text, a byline, a publication date, sometimes a table of contents. In one case the researchers measured, the snippet was entirely table-of-contents text with no content at all. The meta description plays no role at all in this pathway; it's read only by the Google-fed pipelines, where it's used in roughly one of every three results.

The snippet is also frozen at indexing time. RESONEO found the same URL returns the identical snippet regardless of what the user asked, cut once and served as-is until the next crawl updates it. That's a noticeably cruder mechanism than query-dependent snippets, which conventional web search moved past years ago.

The reading cache: shared, stale and blind to noindex

Separately from the discovery index, RESONEO documented a reading cache that keeps a full copy of every page ChatGPT has ever fetched, shared across every user and every tier. If one paying user in one country triggers a fetch of a page, a free user asking about the same page elsewhere gets served that same stored copy rather than a fresh one. The refresh pattern the researchers observed is a stale-while-revalidate model: a copy is treated as current for roughly 30 minutes, after which the next request still gets the stale version immediately while a background fetch quietly updates it for whoever asks next. Since the trigger for a refresh is user interest rather than a schedule, an unpopular page can sit unrefreshed indefinitely. The researchers documented copies still being served more than 90 days after the original fetch, with no eviction limit observed.

Two details about that caching layer stand out. The researchers found Cache-Control: no-store headers and noindex directives were both ignored by this pathway; neither stopped a page from being fetched and stored. And the Markdown conversion applied to cached pages strips scripts, iframes and JSON-LD (JavaScript Object Notation for Linked Data). So structured data, covered in is structured data a ranking factor?, doesn't reach the model through this path, while CSS-hidden text survives the conversion intact. Pages are also capped at exactly 4 MB; past that limit, RESONEO found the fetch rejected outright rather than truncated, so an oversized page isn't partially read, it's not read at all.

Rows of glass jars filled with preserved food stacked on wooden pantry shelves, representing ChatGPT's shared reading cache: full page copies stored and served to later users, some found still in use more than 90 days after the original fetch

The analytics blind spot

OpenAI's own Publishers and Developers documentation confirms that ChatGPT appends the utm_source=chatgpt.com UTM (urchin tracking module) parameter to citation links shown to users, covered in more detail in how to track AI referral traffic in GA4. RESONEO's research adds a wrinkle. Pages ChatGPT opens on its own during thinking mode carry no UTM parameter at all in the citations displayed afterwards, since that tagging applies to the outbound link a user might click, not to ChatGPT's own internal fetch of the page. That means the pages a site owner would most want visibility into, the ones ChatGPT read in full, are exactly the ones a UTM-based tracking setup can't see. RESONEO's guidance is to also watch for the ChatGPT-User agent directly in server logs rather than relying on UTM parameters alone.

Being retrieved vs being cited: the numbers

RESONEO's corpus makes the gap between showing up and mattering concrete. Across their capture, 61,332 URLs were surfaced into the sources shown to users, but only 5,032 became the lead source behind a citation. 759 pages were opened live, 757 of them in paid thinking mode. Of the pages ChatGPT opened and read, 74% ended up cited. Of the pages merely retrieved into the source list without being opened, only 7% were. Being present in the results is a weak signal on its own; being one of the rare pages ChatGPT reads is what correlates with getting named.

One brightly lit window in an otherwise dark apartment building at night, representing how rarely a retrieved page is the one ChatGPT opens and cites out of the many surfaced into its results

How to get cited by ChatGPT: what to change, and what not to overreact to

RESONEO's own recommendations, grounded in what they measured, are modest:

  • Write titles as self-contained statements, not truncation-optimised labels, since the full title reaches the model untruncated in the labrador pathway.
  • Put the substance of a page in the first 200 characters after the H1, since that's the only body text instant-mode answers see.
  • Keep meaningful alt text near the top of a page.
  • Keep the meta description anyway, since Google-fed pipelines still use it.
  • Stay under the 4 MB fetch limit with content that doesn't depend on JavaScript, consistent with the crawler limits covered in do AI crawlers read your site?

The researchers themselves caution against over-indexing on any single mechanism here. The internal field that first let them identify these pipelines disappeared from the data stream overnight partway through their study, and provider names and routing behaviour shifted again within the same reporting window. OpenAI can change the plumbing at any time. So the lasting lesson isn't the exact snippet length or cache duration. It's the underlying shape: a useful, clearly structured, easily crawlable page that answers a real question, whichever pipeline serves a given answer this month.

What's confirmed, what's independent research and what's unknown

This is one of the more detailed pictures of an AI system's internals published so far, but it isn't official documentation. Confirmed directly by OpenAI: that ChatGPT appends utm_source=chatgpt.com to citation links shown to users, and that a ChatGPT-User agent exists and can be identified in server logs. Independent research, not confirmed or denied by OpenAI: the specific pipeline names, the 200-character snippet mechanics, the 30-minute cache freshness window, the 4 MB fetch cap, and the retrieved-versus-cited percentages above. Unresolved even by the researchers' own account: which crawler populates the labrador index, since their own canary pages caught OAI-SearchBot nearly silent while that index kept returning results, and why snippet-less URLs get cited at a higher rate than URLs with snippets in instant mode, a pattern they documented but can't yet fully explain.

Related: do AI crawlers read your site, how to track AI referral traffic in GA4, is structured data a ranking factor?, AEO, GEO and SEO: what each term means.

Frequently asked questions

Has OpenAI confirmed how ChatGPT's retrieval system works?

No. The pipeline names and mechanics described here (labrador, bright, oxylabs, the reading cache) come from independent research by RESONEO, published on its own site in July 2026, based on captured data from real ChatGPT sessions. OpenAI has never documented these internal pipelines publicly and only refers generically to third-party search providers, so treat this as reverse-engineered research, not official confirmation.

Does my meta description matter for getting cited by ChatGPT?

It depends entirely on which retrieval pipeline serves a given answer. RESONEO's research found the meta description is ignored completely by labrador, OpenAI's own index, which powers nearly all of instant mode. It's used about one time in three by the Google-fed pipelines (bright and oxylabs), which mostly appear in the paid, slower thinking mode.

Why doesn't utm_source=chatgpt.com show up for every ChatGPT citation in my analytics?

Per this research, that UTM (urchin tracking module) parameter is appended only to the clickable links displayed to users, the ones OpenAI's own documentation describes. Pages ChatGPT opens and reads on its own in thinking mode carry no UTM trace at all in the citations shown afterwards, even though those opened pages get cited at a far higher rate than pages that are merely retrieved.

Does adding noindex or Cache-Control: no-store stop ChatGPT from caching my page?

According to this research, no. RESONEO documented ChatGPT's reading cache ignoring both signals, serving stored copies of pages more than 90 days after the original fetch with no observed eviction cap. This is independent testing, not a policy OpenAI has stated, but it's worth knowing if you were relying on either directive to keep a page out of ChatGPT's hands.