A page is usually missing from Google for one of four reasons: Google has not found the page yet, an instruction such as noindex or a block in robots.txt holds it back, Google picks another URL as the main version, or Google does not consider the page valuable enough to index. URL Inspection in Google Search Console shows which of those causes applies to your page.
This guide works through the causes in order. You need access to Google Search Console for your website. Work from top to bottom: the first steps are the quickest to check and rule out the most common technical causes.
How to find out why a page is missing
- Really missing?Search with
site:and open URL Inspection in Search Console. - New?Google may not have fetched a new page yet.
- Noindex?An instruction in the HTML or the HTTP header asks Google not to index the page.
- Robots.txt?The page is not allowed to be fetched.
- Another main version?Google picks a different URL as the canonical version.
- Status code?The page returns an error, a redirect or a login screen.
- Skipped?Everything is technically fine, but Google is not indexing the page for now.
Step 1: Check whether the page really is missing
Search Google for site:yourdomain.com/path-of-the-page. If the page appears, it is in the index and the problem is more likely its position than its indexing.
site:example.com/e-bikes/maintenance
- example.com › e-bikes › maintenance* E-bike maintenance: what you can do yourselfBattery, chain and brakes: how to keep your e-bike in good shape.
- example.com › e-bikes › maintenance › workshopBook an e-bike serviceSchedule a service at our workshop.
A site: search is a quick first check, not a complete overview. If the page is missing, Search Console gives the most reliable answer.
If it does not appear, open Search Console and paste the full URL into the search bar at the top. That opens URL Inspection. Google shows one of three outcomes: URL is on Google, URL is on Google but has issues, or URL is not on Google. Under Page indexing you see why.
Bear in mind that URL Inspection shows the version Google indexed most recently. If you have just changed something, click Test live URL to check the current version.
Step 2: Is the page new?
Google may not have fetched a page that is only a few days old. According to Google, crawling can take anywhere from a few days to a few weeks.
What to do:
- Link to the new page from an existing page that gets plenty of visits. Google finds new pages mainly through links.
- Make sure the page is in your XML sitemap.
- Click Request indexing once in URL Inspection. Google says repeated requests for the same URL do not make it go faster.
Step 3: Is there a noindex instruction?
In URL Inspection, look at Indexing allowed? If it says “No”, the page is asking Google not to index it. That noindex instruction can sit in two places:
- In the HTML, as a meta robots tag in the
<head>. You find it by opening the source code and searching fornoindex. - In the HTTP header of the server response, as an X-Robots-Tag. You will not see this in the source code. Open your browser’s developer tools, go to the Network tab, reload the page and look at the response headers of the first request.
<head>
<title>E-bike maintenance</title>
<meta name="robots" content="noindex">
</head>HTTP/2 200
content-type: text/html; charset=utf-8
x-robots-tag: noindexThe header version is often missed, because the page looks normal in the source code. It tends to turn up when a setting from a test environment has been carried over to the live site.
What that finding looks like in Atlas
Atlas, developed by Outseen, also reads the HTTP headers a page returns. This is a real check on Outseen’s own test site:
| Field | Observation |
|---|---|
| Check | May this page appear in search results? |
| Source | HTTP header X-Robots-Tag |
| Observed | noindex |
| Outcome | Needs attention |
| Measured | 11 November 2026 |
| Engine | atlas-engine/0.3.12 |
When a finding like this turns up on a client’s website, a specialist checks whether the page belongs in search results. If the block is unintended, it is removed and Atlas checks the page again. This check says something about indexing, not about your position in Google or about mentions in AI answers.
Besides noindex, Atlas checks status codes, robots rules and canonicals on the pages it fetches, together with what their target page actually returns. Every finding states what was observed, in which source, on which URL and when. For a first impression of a limited number of technical points, there is the website check. It says nothing about rankings or AI mentions.
Step 4: Does robots.txt block the page?
In URL Inspection, look at Crawl allowed? If it says “No”, your robots.txt excludes the page. Open yourdomain.com/robots.txt and find the Disallow rule that matches the page’s path.
Two things to know:
- According to Google, robots.txt is not a way to keep a page out of Google. A blocked URL can appear in the results without a description if other pages link to it.
- If a page has both a noindex and a robots.txt block, Google does not see the noindex, because Googlebot is not allowed to fetch the page.
Step 5: Does Google pick another URL as the main version?
When the same or nearly the same content sits at several addresses, Google picks one as the canonical URL and leaves the rest out. In URL Inspection, compare the User-declared canonical with the Google-selected canonical.
If they differ, go through your own signals. Google looks at all of these places at once.
Do all your signals point to the same URL?
- The canonical in your source code points to the page itself, as a full URL
- Your internal links point to the same version, not to a variant with a parameter or a different spelling
- Your XML sitemap contains the same URL
- An old or duplicate version redirects with a 301 to the right URL
Google says the canonical is a strong signal, not a directive. If your signals contradict each other, Google chooses for itself.
Step 6: Does the page return the right status code?
A page that should be found returns status code 200. The Page indexing report in Search Console lists the other cases separately:
| Reason in Search Console | What is going on | What you do |
|---|---|---|
| Not found (404) | The server says the page does not exist | Restore the page, or redirect with a 301 to the best replacement |
| Soft 404 | The page returns 200 but looks like an error page or is empty | Give the page real content, or let it return a 404 |
| Page with redirect | The URL redirects; Google indexes only the target | No action if that is intended. Link internally straight to the target |
| Server error (5xx) | The server returned an error when Google fetched the page | Ask your hosting provider or developer to find the cause |
| Blocked due to unauthorized request (401) | The page asks visitors to log in | Remove the login requirement if the page should be public |
Google does not use content from pages that return a 4xx code and removes them from the index.
Step 7: Has Google skipped the page?
If all the instructions are in order and the page is still not in the index, the Page indexing report often shows one of these two reasons:
- Crawled - currently not indexed. Google fetched the page and decided not to index it for now. Google calls this a common message and says it does not necessarily mean something needs attention.
- Discovered - currently not indexed. Google knows the URL but has not fetched it yet, for example to avoid overloading your server.
There is no technical switch to flip here. In its guide to how Search works, Google says that not every page it processes gets indexed. Questions that help:
- Does the page add something that is not already on your own site, or is it very similar to another page?
- Does the page answer a question people actually ask?
- Do your other pages link to it, or does it stand apart from the rest of your site?
Improving the page, merging it with a similar page or linking it better from within your site works more often here than requesting indexing again.
Step 8: What do you do after the fix?
After the fix
- Test liveUse Test live URL to check that the cause has gone.
- Request onceClick Request indexing once. According to Google, repeating it does not make it faster.
- Follow upKeep an eye on the Page indexing report. After a noindex change, Google has to fetch the page again first.
If the problem affects more pages, work through the full technical SEO checklist. How crawling and indexing fit together is explained in what is SEO and how does it work?
What does this mean for your business?
A missing page almost always has an identifiable cause, and Search Console usually tells you which one. Check the pages that matter for your revenue first, such as your main services, categories or products, and in any case do so after a new website, a move to another domain or a major CMS update. Those are exactly the moments when settings from a test environment get carried over to the live site.
If the instructions are right and a page still stays out of the index, look at the content. A page that adds little to the rest of your site is better merged or expanded than submitted again and again. How the index relates to Google’s AI features is covered in AI Overviews and AI Mode.