Skip to content
Kuraib Ali

AI search guides

How AI systems decide what to read, trust and cite, covering answer engine optimisation (AEO), generative engine optimisation (GEO) and AI Overviews.

By Kuraib AliPublished Updated

AI search means a system reads across multiple sources and gives a synthesised answer, instead of returning a list of links for a person to click through. Google's AI Overviews, ChatGPT, and Perplexity each do a version of this. The mechanics differ, but the underlying question is the same for all of them: can this system access the page, understand what it says, and trust it enough to use it as a source.

That's a different bar than traditional ranking. A page can rank well in classic search and still never get pulled into an AI-generated answer, because AI systems apply their own filtering on top of, not instead of, normal SEO fundamentals.

AEO. Answer Engine Optimisation. Structuring content so an AI system can lift out a direct, accurate answer without interpreting an entire page. Mostly a writing and structure problem: does the page state the answer clearly, near the top, in a form that can stand alone.

GEO. Generative Engine Optimisation. Making sure AI systems can physically access and read a page, and have a reason to trust it as a source. Closer to a technical-access problem: crawler permissions, rendering, and consistent, verifiable authorship.

AI Overviews & AI Mode. Google's specific AI-generated result formats. Both draw from multiple sources at once and decide, per query, whether to show an AI answer above or instead of a normal results list.

AI citations. What gets a page referenced inside an AI-generated answer: not just used silently in the background, but explicitly named or linked as a source.

An abstract graphical user interface, representing the layer AI search systems build on top of a page's underlying content

Access comes before persuasion

A page an AI system can't read at all will never get cited, no matter how well-written the answer is inside it. Check GEO fundamentals (crawler access, rendering) before spending time optimising AEO-style answer structure.

A glowing smart speaker, representing the voice and AI assistants now retrieving and citing web content directly

How AI systems retrieve and cite content

The mechanism differs by product, but the general pipeline behind most AI-generated answers has three stages, and a failure at any one of them means the page never gets used, regardless of how good the content is:

  1. Crawl access. Each AI operator runs its own named crawler: OpenAI's GPTBot and OAI-SearchBot, Anthropic's ClaudeBot, Perplexity's PerplexityBot. These are independent from Googlebot and from each other; a site can allow Googlebot while unintentionally blocking every AI crawler, or vice versa. Google-Extended and Applebot-Extended aren't crawlers at all. They're permission flags on content already fetched by Googlebot and Applebot. Google-Extended controls whether content can be used for Gemini apps and Vertex AI. Google says it does not affect a site's inclusion in Google Search, so it does not control AI Overviews or AI Mode. See robots.txt and AI crawlers for the full breakdown, since this is the single most commonly misconfigured piece.
  2. Rendering. A crawler that fetches a page's raw HTML still needs the actual answer-bearing content to be present in what it fetches. Content injected client-side after page load, with no server-rendered fallback, can be invisible to crawlers that don't execute JavaScript the same way a browser does. See JavaScript SEO for exactly what Googlebot renders and what several AI crawlers don't.
  3. Selection and citation. Once content is accessible and rendered, it competes with every other accessible source for inclusion in a generated answer. This is where AEO-style structure (a direct answer near the top, clear entity naming, content that stands alone as a citable unit) matters. It's also where signals related to E-E-A-T (experience, expertise, authoritativeness and trust), such as consistent authorship and verifiable claims, plausibly influence which of several accessible sources gets picked.

A brass balance scale, representing the weighing of what's verified in AEO/GEO advice against what's still unsettled

What is verified and what is still unsettled

A lot of AEO and GEO advice states things with more confidence than the evidence supports.

Verified, from the operators' own documentation: which crawlers exist and what each one is for (training vs. search-index vs. live user fetch); that Google-Extended and Applebot-Extended are permission flags, not crawlers; that AI Overviews and AI Mode draw from multiple sources and apply filtering on top of normal ranking, not instead of it; that Search Console now has a Generative AI performance report, separate from the standard Search performance report.

Reasoned inference, not directly confirmed by any operator: that clear, front-loaded answers and well-structured tables improve citation odds. This is a reasonable extrapolation from how retrieval systems work, and it fits what AEO practitioners report. But no operator has published a controlled study measuring it. Treat any exact citation-rate lift claimed for a particular technique with scepticism.

Unsettled: how much weight, if any, each AI system gives to traditional backlink authority versus content structure versus freshness when selecting sources for a specific answer. It varies by system, likely changes without announcement, and no outside party can measure it precisely. Treat any page that states a specific ranking weight for AI citations, mine included, as making a claim it can't back up.

Ten guides and one research note are published so far:

Looking for the technical-SEO side of this: crawling, indexing, Core Web Vitals? See SEO.

Frequently asked questions

What's the difference between AEO and GEO?

AEO (Answer Engine Optimisation) is a writing and structure problem: does the page state a direct, accurate answer clearly enough that an AI system can lift it out without interpreting the whole page. GEO (Generative Engine Optimisation) is closer to a technical-access problem: can AI crawlers reach and render the page, and is there enough verifiable authorship for the system to trust it as a source. A page can be excellent on one and fail on the other.

If I block AI crawlers, does that affect my Google ranking?

Not directly. Google's ordinary ranking and indexing runs on Googlebot, which is separate from Google-Extended (the AI-training permission flag) and from other operators' crawlers like GPTBot and ClaudeBot. Blocking Google-Extended, GPTBot, or ClaudeBot doesn't touch standard Google Search rankings. It only affects whether that content can be used to train or ground those specific AI products. See robots.txt and AI crawlers for exactly which rule controls what.

Does having good SEO automatically mean good AI search visibility?

No, but it's a prerequisite, not a substitute. A page that's crawlable, well-structured, and already ranking well has cleared most of the access barrier AI systems apply. Beyond that, AI systems layer additional filtering on top of ordinary ranking signals, which is why a page can rank normally in Google and still never get pulled into an AI-generated answer.

Can I check whether AI crawlers are visiting my site?

Yes, the same way you'd check for any bot: server log analysis showing requests from user agents like GPTBot, ClaudeBot, or PerplexityBot. A crawler being allowed in robots.txt doesn't guarantee it's visiting. Log files are the only direct evidence of real crawl activity, the same principle covered in crawl budget and log files for traditional SEO.

Is AI Overviews the same thing as a featured snippet?

No. A featured snippet pulls one excerpt from a single source page. AI Overviews synthesise an answer across multiple sources at once and decide, per query, whether to show that synthesised answer above or instead of the normal results list. A page can be well-optimised for featured snippets and still not be one of the sources an AI Overview chooses to draw from.