A search engine finds web pages, stores what they contain in an index and picks the best matches when someone searches. Google and Bing both work this way. Google describes the three stages in its guide to how Google Search works:
- Crawling: programs fetch pages the search engine has found.
- Indexing: the search engine analyses the content and stores it in the index, a large database.
- Serving results: for each search, the search engine picks what best fits the question from the index.
There is no central register of all web pages. Google therefore keeps looking for new and changed pages, for example through links on known pages and through sitemaps that site owners submit.
Why does it matter?
A page that a search engine cannot fetch or does not add to its index cannot appear as a result. That also applies to AI features within search engines: Google writes in AI features and your website that a page must be indexed to be eligible for AI Overviews or AI Mode.
Example
Suppose you publish a new page about solar panels on a flat roof, but no other page links to it and it is not in your sitemap. A search engine may then not find it.
Common mistake
Assuming a search engine includes every page on your site. Google writes that even a page that meets all requirements is not guaranteed to be crawled, indexed or shown.
More in What is SEO and how does it work? and Why is my page not showing in Google?.