Why Google Is Not Indexing Your Pages (And How to Read Search Console)

Submitting a sitemap does not get pages indexed. What "discovered" and "crawled – currently not indexed" actually mean, and why blocking a page keeps it in the index.

Submitting a sitemap does not get your pages indexed. This surprises people, and it is the root of most "Google is ignoring my site" panic. A sitemap is a suggestion — it tells Google which URLs exist and when they changed. Whether any of them earn a place in the index is a separate decision, made later, on different grounds.

Google Search Console will tell you which decision was made about each page, if you know where to look and what the wording means. The Pages report under Indexing lists every URL Google knows about, grouped by status, and those status labels are far more specific than they first appear.

Discovered versus crawled — a crucial difference

Two statuses sound almost identical and mean completely different things.

Discovered – currently not indexed means Google knows the URL exists but has not fetched it yet. It found the link, or read it in your sitemap, and put it in a queue. Nothing has been evaluated. This is normal for new pages and for large sites publishing faster than Google chooses to crawl. If it persists for weeks across many URLs, it usually points at crawl capacity — a slow server, or so many URLs that Google is rationing its attention.

Crawled – currently not indexed is the one that actually stings. Google fetched the page, read it, evaluated it, and decided not to index it. That is a quality judgement. There is no technical fault to fix and no button to press. The page was found wanting compared to what is already in the index for those queries.

Confusing these two wastes enormous amounts of time. People see "not indexed" on a crawled page and start auditing their robots.txt, when the crawler clearly had no trouble reaching it. The fix for one is technical; the fix for the other is the content.

Blocked is not the same as excluded

Here is a subtlety that catches out even experienced people: robots.txt does not prevent indexing. It prevents crawling.

If Google cannot fetch a page but finds links pointing at it, it may index the URL anyway, with no description — the "indexed, though blocked by robots.txt" status. It knows the page exists and roughly what it is about from the link text, but has never read it.

Worse, the two directives conflict in a way that is genuinely counterintuitive. If you want a page kept out of the index, you use a noindex meta tag. But Google has to crawl the page to see that tag. Block the page in robots.txt and Google never reads the noindex, so the URL can sit in the index indefinitely. To remove a page properly, allow crawling and serve noindex — the opposite of what instinct suggests.

The self-inflicted causes, in the order they usually occur

Most indexing problems on small sites are things the site did to itself, and they are worth checking before concluding Google has judged your content.

A stray noindex tag is the most common by a distance, and it is usually left over from staging. A site built behind a noindex, deployed to production with the tag still in place, will never be indexed and gives no obvious symptom. Check the live HTML, not the template.

A canonical pointing somewhere else tells Google the page is a duplicate and the other URL is the real one. Google will generally honour it and index the target instead. Misconfigured canonicals — every page canonicalising to the homepage is a classic — can silently deindex an entire site. The Search Console status is "Alternate page with proper canonical tag", which sounds reassuring and is not, if the canonical is wrong.

Soft 404s happen when a page returns HTTP 200 but reads as empty or missing — a search results page with no results, a product page for a discontinued item that says nothing else, a category with no entries. Google classifies it as effectively a 404 and declines to index.

Blocked resources are subtler. If your CSS and JavaScript are disallowed, Google renders the page without them. On a JavaScript-heavy site this can mean Google sees an empty shell, judges it thin, and moves on. Blocking a framework's asset directory is a surprisingly easy mistake to make and an expensive one.

What crawl budget actually is, and why it probably is not your problem

Crawl budget gets invoked constantly and applies to very few sites. It is the intersection of how much crawling your server can tolerate and how much Google thinks your content warrants. For a site under a few thousand URLs, it is effectively never the limiting factor, and optimising for it is a distraction.

Where it does matter is on large sites generating many low-value URLs — faceted navigation producing every combination of filters, calendar pages extending indefinitely, session parameters creating infinite variants of the same page. There, Google can spend its attention on junk and never reach the pages you care about. The fix is to stop generating the junk, not to ask for more crawling.

What to do about "crawled, currently not indexed"

Since this is a quality judgement, the responses that work are the ones that change the judgement.

Ask honestly what the page offers that the pages currently ranking do not. If the answer is "the same information, worded differently", that is precisely the assessment Google made. Thin pages, pages that are near-duplicates of each other, and pages generated from a template at scale are the usual occupants of this status.

Internal linking matters more than people expect. A page reachable only from a sitemap, with no links from anywhere on your own site, signals that you do not consider it important either. Pages linked from relevant places in your content get crawled more often and indexed more readily.

And consolidation frequently beats creation. Three thin pages competing for the same topic often do worse than one substantial page covering it properly. Merging them, and redirecting the old URLs, concentrates the signals instead of splitting them.

Requesting indexing, and the wait

The URL Inspection tool has a Request Indexing button, with a small daily quota. It is genuinely useful for a handful of important pages — a new landing page, a page you have just substantially rewritten. It is not a way to push a site into the index, and using it on hundreds of URLs achieves nothing.

Two expectations worth setting. Removing a noindex tag does not reindex a page immediately; Google has to crawl it again, which can take days or weeks. And the Pages report in Search Console updates on a delay, so a fix you deployed today will not show there tomorrow. Verify the fix with URL Inspection's live test, which fetches the page right now, and use the report to confirm the trend later.