AI assistants choose sources in one of two ways. The language model answers from what it learned during training, or the assistant searches the web live and bases its answer on the pages it finds. Only with live search can an AI assistant link to your website. Which pages it then mentions depends on the question, the search results it retrieves and the platform.
Training data or live search?
A large language model learns from a large collection of text. What it contains is knowledge up to a certain date: the knowledge cutoff. If you ask something and the assistant does not search, the answer comes from that stored knowledge. There is usually no source link, and recent changes to your website do not count.
Many assistants can also search first. That is called retrieval: the assistant fetches current pages and uses them as the basis for its answer. The technical term for this is retrieval-augmented generation. That is when your website can appear as a source.
Two ways an AI assistant answers
| From training data | With live search | |
|---|---|---|
| Where does the answer come from? | Knowledge the model picked up during training | Pages the assistant fetches at the moment of the question |
| How current is it? | Up to the knowledge cutoff | As current as the pages it finds |
| Link to your website? | Usually not | Possible, if your page is among the sources |
| Which bots are involved? | Training bots, such as GPTBot and ClaudeBot | Search bots, such as OAI-SearchBot, Claude-SearchBot and PerplexityBot |
| Does a change on your site count? | Only in a later model | Once the page has been fetched again |
AI companies often use separate AI crawlers for these two purposes. In its crawler documentation, OpenAI distinguishes GPTBot, for content that may be used to train models, from OAI-SearchBot, which makes websites appear in ChatGPT’s search feature. Anthropic’s crawler documentation lists ClaudeBot for training, Claude-SearchBot for search results and Claude-User for pages a user asks for. Perplexity says PerplexityBot surfaces and links websites in its search results and is not used to train models (Perplexity Crawlers). How to allow or block these bots separately is covered in Allowing or blocking AI crawlers with robots.txt.
What is query fan-out?
With query fan-out, a system splits one question into several related searches. In AI features and your website, Google describes how AI Overviews and AI Mode may use this technique: the system searches across subtopics and different data sources at the same time. As a result, Google can show a wider range of pages as links than with an ordinary search.
How one question can split into sub-searches
- QuestionSomeone asks: which e-bike suits a 25 kilometre commute?
- Sub-questionsThe system searches separately for range, batteries, frame size and maintenance.
- SearchFor each sub-question it fetches the pages that fit best.
- AnswerIt writes one answer and may cite a different source for each part.
For you, that means a page which answers one part of the question well can appear as a source, even if it does not cover the whole question. A bike shop with a clear page on battery range could be cited for a broad question about e-bikes. See also the glossary entry query fan-out.
What is grounding?
Grounding means a language model receives information from an external source while it answers, so the answer rests on current facts rather than training knowledge alone. In its documentation on Google-Extended, Google describes it as content from the Google Search index that is given to the model at the moment of the question. With the robots.txt token Google-Extended you decide whether Google may use your content for this in Gemini apps. According to Google, that setting has no effect on your inclusion or ranking in Google Search.
What do the platforms say themselves?
The platforms describe the process, not the exact weighting. This is what the official documentation says:
| Platform | When does it search? | How does it show sources? | What does access require? |
|---|---|---|---|
| Google AI Overviews and AI Mode | AI Overviews only appear when Google considers them useful; AI Mode is meant for questions that need more depth | Links to supporting pages alongside the answer | The page is indexed and eligible for a snippet; Googlebot is not blocked |
| ChatGPT | In OpenAI’s developer documentation, the model decides for itself whether to search, depending on the question | Inline citations with a link and title | OAI-SearchBot allowed in robots.txt |
| Perplexity | Searches in real time, according to Perplexity | Numbered citations with every answer | PerplexityBot allowed in robots.txt |
| Claude | Searches when a question needs current information, or when the user asks it to | Citations with the answer | According to Anthropic, blocking Claude-SearchBot or Claude-User can reduce your visibility in search answers |
In practice, a citation like that can look like this:
Where can I get my e-bike serviced in York?
For e-bike servicing, look for a bike workshop with experience of electric drive systems. The Spoke Cycle Shop in York lists a regular e-bike service on its website and checks the battery and motor software as part of it. Ask any workshop which motor brands it supports.
Bronnen: thespoke.example.com, example.com
How this works out on each platform is covered in How do you become visible in ChatGPT? and AI Overviews and AI Mode.
Why do answers differ by question, moment and platform?
Several things are going on at once.
- Different models and techniques. Google says AI Overviews and AI Mode may use different models and techniques, so the links they show differ. Other platforms use their own models and search systems.
- Different searches. A small change to the question produces different sub-searches, and therefore different sources.
- Changing search results. Pages are added, removed or updated. An answer from last week relied on last week’s results.
- The user’s context. Anthropic says Claude may use an approximate location for local results. Two people asking the same question will not always get the same answer.
That is why a single answer tells you little. If you want to know how often your business is mentioned, ask the same questions repeatedly and record the answers. How to set that up is explained in How do you measure your visibility in AI answers?
Checking one customer question yourself
- Write the question down the way a customer would ask it
- Ask it on each platform you want to follow, on a few different days
- Note the date, the platform and whether the assistant searched
- Note which businesses and which source links the answer mentions
- Check whether the cited pages are yours, or another site writing about you
- If something is wrong, save the answer with its date
What can you influence yourself?
You have no say over the selection rules, but you do control what the assistant can find:
- Access. Allow the search bots of the platforms where you want to be visible, and make a deliberate choice about training bots.
- Readable text. Put important content as text in the HTML your server sends. Not every bot runs JavaScript, so text that only appears in the browser may be missing. More in Can AI crawlers read your website?
- One page per question. A page that clearly answers one customer question fits a sub-search better than a page that touches on everything.
- Consistent information. Make sure your business details match on your site and elsewhere, so an assistant does not find sources that contradict each other.
There is no such thing as a guaranteed mention: each platform decides for itself what it shows. How to build up this work is explained in What is GEO?
What does this mean for your business?
Start with the bots and the text, because you can check those today. Look in your robots.txt to see whether the search bots of ChatGPT, Claude and Perplexity may fetch your pages, and open your most important pages with JavaScript switched off to see whether the text is still there. A page a bot cannot fetch or read cannot become a source.
Then pick a handful of questions customers genuinely ask before they buy from you, and check whether you have a page that answers each one. Where that page is missing, it is the first one to write. Ask the same questions again every month and compare the answers; that way you see whether anything changes, without one chance answer shaping your view.