Glossary

Indexing

Written by
Outseen
Updated

Short answer

Indexing is the step in which a search engine analyses a fetched page and stores the information in its index, the database search results come from. Only indexed pages can appear in search results, including AI Overviews and AI Mode.

Indexing is the step in which a search engine works out what a page is about and decides whether to store it. After crawling, Google groups pages with similar content and picks the most representative page in each group, the canonical, according to its guide to how Google Search works.

Along the way, Google collects signals such as the page’s language, the country the content targets and its usability. Not every crawled page is included.

Why does it matter?

The index is the stock Google chooses from. Google writes in AI features and your website that AI Overviews and AI Mode also only use pages that are indexed and eligible to show a text snippet.

Example

A page you deliberately want to keep out of the index gets a noindex rule:

<meta name="robots" content="noindex">

How do you check it?

Use URL Inspection in Google Search Console. You can see whether the page is in the index, which canonical Google chose and whether there is a reason the page was not included.

Common mistake

Combining noindex with a block in robots.txt. Google writes in its noindex documentation that the crawler then never sees the noindex rule. The page can still appear in results if other pages link to it.

More in Why is my page not showing in Google? and under noindex.

Sources

Read next