If you have ever exported your Google Search Console data and found hundreds of URLs sitting under crawled currently not indexed, you already know the frustrating part: Google visited the page, read it, and then walked away without adding it to the index. No error. No penalty. Just silence.
This guide is a diagnostic walkthrough. We go status by status through the Pages report (the report most people still call the Index Coverage report), explain what each exclusion reason actually signals, and give you the concrete first action for each case. By the end you will know exactly which of your URLs are excluded, why, and which fix deserves your time this week.
Where to find the report (and what it really shows)
In Google Search Console, open Indexing > Pages. You get two buckets:
- Indexed: pages eligible to appear in Google Search.
- Not indexed: pages Google knows about but has chosen not to index, grouped by reason.
Two things people misread constantly:
- “Not indexed” is not automatically a problem. Redirects, canonicalised duplicates and noindexed tag archives all land here by design. A healthy site has a large “not indexed” number.
- The list is capped. Each reason shows a sample of up to roughly 1,000 example URLs. If a reason affects 40,000 URLs, you are looking at a sample, not an inventory. For a full picture, combine the report with a crawl of your own site (Screaming Frog, Sitebulb) and the URL Inspection API, which lets you check indexing status in bulk.
Read the report in triage order, not top to bottom
The report sorts by volume, which is not the same as sorting by damage. Use this priority order instead:
| Priority | Status | Why it matters |
|---|---|---|
| 1 | Excluded by noindex tag / Blocked by robots.txt | Accidental blocks kill money pages instantly. Check first, always. |
| 2 | Soft 404 | Signals Google sees your content as empty or worthless. |
| 3 | Duplicate without user-selected canonical | Google is guessing which version to keep. You should be deciding. |
| 4 | Crawled – currently not indexed | A quality and demand judgement. Fixable, but slowest to resolve. |
| 5 | Discovered – currently not indexed | Crawl capacity and site architecture issue. |
| 6 | Alternate page with canonical / Page with redirect | Usually correct. Spot check only. |
Status 1: Crawled – currently not indexed
Crawled – currently not indexed means Googlebot fetched the URL, processed the content, and decided that indexing it was not worth the storage and serving cost. There is no technical blocker. Google simply was not convinced.
The word currently is doing real work in that label. It is a provisional verdict, not a ban. Pages move out of this bucket regularly once the underlying signal changes.
The real causes, ranked by how often we see them
- Near-duplicate content at scale. Location pages, product variants, filtered category pages, or programmatically generated articles where 80% of the text repeats. Google indexes one, ignores the rest.
- Thin or derivative content. The page answers the query no better than the ten results already indexed. Common with 300-word blog posts, tag archives, and AI-spun content published without editing or first-hand input.
- No internal link support. Orphan pages that exist only in the sitemap tell Google the site owner does not consider them important either.
- Low overall site quality signals. On sites with a weak quality profile, Google indexes selectively. Individual good pages get caught in the net.
- Zero search demand. No query exists for the content. Google has no reason to store it.
- Very new URL or very new site. On a young domain, delays of several weeks are normal.
- Rendering problems. Content that only appears after a client-side JavaScript fetch may be indexed as a near-empty shell.
The fix, in the order you should do it
- Export and segment. Download the sample URLs as CSV and sort them by template: blog, product, category, tag, pagination, search pages, parameters. Patterns appear immediately. In most audits, 70% or more of the URLs belong to a single template.
- Decide which URLs you actually want indexed. This is the step people skip. Internal search results, faceted filters, paginated pages 2+, thin tag archives and author pages usually should not be indexed. If a page is in this bucket and you do not want it ranked, the correct action is to remove it, noindex it, or canonicalise it and move on. Do nothing else.
- Check the rendered HTML. Use URL Inspection, then View crawled page, and read the HTML Google actually received. If your main content is missing, you have a rendering problem, not a quality problem.
- Compare against what ranks. Take the target query for the page. If the indexed results offer original data, depth, or first-hand experience your page does not, that is the gap to close. Merging four weak posts into one strong page works far better than editing four weak posts.
- Add internal links from indexed, crawled pages. Three to five contextual links from strong pages on the same topic is the single most reliable lever we have for pushing a page out of this status.
- Refresh the content substantially, then request indexing once. Substantially means new sections, new examples, new data, not a date change. Then use Request Indexing in URL Inspection for the priority URLs only.
- Wait, then re-check. Two to six weeks is a normal window. Re-hitting Request Indexing daily changes nothing.
What does not work
- Resubmitting the same sitemap repeatedly.
- Changing the publish date without changing the content.
- Adding schema markup to a thin page. Structured data does not create value.
- Buying indexing services that ping URLs. Google ignores them.
Status 2: Discovered – currently not indexed
Google found the URL, usually via a sitemap or an internal link, but has not crawled it yet. This is a different problem with a different fix. It is about crawl capacity and site structure, not content quality.
Typical causes:
- The site has more URLs than its crawl allocation supports, often because of parameter explosion or a filtered navigation generating millions of combinations.
- Slow server response. If time to first byte is high or the server returns 5xx under load, Googlebot backs off.
- URLs buried five or more clicks deep with no supporting links.
- A sitemap containing tens of thousands of low-value URLs, which dilutes the signal.
The fix:
- Open Settings > Crawl stats. Look at average response time and the breakdown by response code and file type. High average response time is your first lever.
- Cut the URL count Google has to consider. Block parameter combinations and internal search in robots.txt, remove faceted URLs from the sitemap, and stop linking to them.
- Flatten architecture. Add hub pages, improve pagination, and link to important deep URLs from category pages.
- Split the sitemap by section so you can see which segments are actually getting crawled, and keep only canonical, indexable, 200-status URLs in it.
Status 3: Duplicate without user-selected canonical
Google found several URLs with essentially the same content and no clear canonical instruction, so it picked one itself and excluded the others. A related status, Duplicate, Google chose different canonical than user, means you did declare a canonical but Google overruled it.
Where it comes from: tracking parameters (?utm_, ?ref=), session IDs, HTTP and HTTPS versions, www and non-www, trailing slash variants, uppercase and lowercase paths, print versions, and the same product living under multiple category paths.
The fix:
- Use URL Inspection on an affected URL and read Google-selected canonical versus User-declared canonical. That single comparison tells you whether the problem is a missing instruction or a rejected one.
- Add a self-referencing canonical to every indexable page, and point every variant at the preferred version.
- 301 redirect protocol and hostname variants so only one form is reachable.
- If Google is ignoring your canonical, the two pages are probably not different enough to justify two pages. Either merge them, or genuinely differentiate the content, titles, headings and internal linking.
- Keep signals consistent. Canonical, internal links, sitemap entry and hreflang must all point to the same URL. Contradictions are the main reason Google overrides a canonical.
Status 4: Soft 404
The URL returns a 200 OK status but Google concluded the page is effectively empty, missing, or an error message. Google trusts what it sees over what your server claims.
Common triggers:
- “No results found”, “Out of stock”, “This product is no longer available” pages served with a 200.
- Empty category, tag or archive pages with zero items.
- A JavaScript app that returns the shell HTML for a route that no longer exists.
- Pages with almost no unique text, such as a single image or a bare form.
The fix depends on intent:
| Situation | Correct action |
|---|---|
| Content is genuinely gone for good | Return a real 404 or 410 |
| A close equivalent exists | 301 redirect to that page, not to the homepage |
| Product temporarily out of stock | Keep 200, keep full product content, add availability schema and alternatives |
| Empty archive or filter page | Noindex it while empty, or stop generating it |
| Page has real content but very little text | Expand the unique on-page content and check rendering |
Status 5: The statuses that are usually fine
Alternate page with proper canonical tag
Working as intended. Your canonical was accepted and this variant was folded into the main URL. Only investigate if a URL you want ranked appears here, which means it is canonicalising to something else by mistake.
Page with redirect
Normal. Spot check for redirect chains longer than two hops and for redirects that land on irrelevant pages.
Excluded by noindex tag
Intentional most of the time. The danger is silent accidents: a staging noindex pushed to production, a plugin setting, or a template-level tag. Filter this list for URLs that should rank. Fixing one accidentally noindexed category page often recovers more traffic than months of content work.
Blocked by robots.txt
Remember that robots.txt blocks crawling, not indexing. A blocked URL can still appear in search results if it is linked elsewhere. If you want a page out of the index, allow crawling and use noindex instead.
Not found (404) and Server error (5xx)
404s from old, genuinely removed content are fine. What matters is 404s that are still internally linked or listed in your sitemap. 5xx errors are always worth immediate attention because they suppress crawl rate site-wide.
Crawled anomaly and Blocked due to unauthorized request (401)
Check for aggressive bot protection, firewall rules, rate limiting or geo-blocking that is refusing Googlebot. Verify with a live test in URL Inspection.
A 30 minute triage workflow
- Open Indexing > Pages and note the indexed to not-indexed ratio.
- Filter to All submitted pages. Anything in your sitemap that is not indexed is a direct contradiction you created, and that is your shortlist.
- Scan Excluded by noindex and Blocked by robots.txt for pages that should rank. Fix immediately.
- Export crawled currently not indexed and group by URL template.
- For each template, answer one question: do I want this indexed? Noindex or remove the ones you do not want.
- For the ones you do want, run URL Inspection on three samples and check the rendered HTML and Google-selected canonical.
- Apply the fix (merge, expand, internally link), then click Validate Fix at the status level.
- Diarise a review 21 days later. Do not touch it before then.
Validation, and why it says “failed”
When you click Validate Fix, Google re-crawls the sample URLs in batches. Validation can take a few days to several weeks and it is common to see Validation failed even when your fix was correct.
Two reasons for that:
- Validation only needs a handful of sample URLs to still show the issue for the whole run to be marked failed. If most URLs passed, you made progress.
- For crawled currently not indexed, there is no binary fix to validate. Google has to reassess quality, and that takes longer than a validation window.
Judge success by the indexed count trend over 4 to 8 weeks and by impressions in the Performance report, not by the validation label.
Benchmarks: when to worry
| Signal | Probably fine | Investigate |
|---|---|---|
| Submitted sitemap URLs indexed | Above 90% | Below 70% |
| Time in “crawled – currently not indexed” for a new post | Up to 3 weeks | Over 2 months |
| Soft 404 count | A handful | Growing week over week |
| Duplicate without canonical | Parameter URLs only | Core templates appearing |
FAQ
How do I fix “crawled – currently not indexed”?
Decide first whether you want the URL indexed. If yes, close the value gap against the pages that currently rank, make sure the content renders in the HTML Google receives, add internal links from strong indexed pages, deduplicate near-identical templates, then request indexing once and allow two to six weeks. If no, noindex it or remove it.
Is “crawled – currently not indexed” a penalty?
No. It is a cost and quality decision, not a manual action. Manual actions appear in the Security & Manual Actions section of Search Console, and this status never appears there.
What is the difference between discovered and crawled currently not indexed?
Discovered means Google has not fetched the page yet, which is a crawl capacity or architecture issue. Crawled means Google fetched it and chose not to index, which is a value, duplication or rendering issue.
Does requesting indexing actually help?
It queues the URL for a faster crawl. It does not force indexing and it does not improve your odds if the underlying reason has not changed. It is most useful right after you ship a real fix. Much the same conclusion turns up on mariehaynes.com.
How long does Google take to index a page?
On an established, frequently crawled site, hours to a few days. On a newer domain or a deep URL with little internal link support, several weeks. If nothing has changed after two months, treat it as a diagnosis to make rather than a delay to wait out.
Can a page rank if it says “crawled – currently not indexed”?
No. Indexing is a prerequisite for ranking. If the URL shows impressions in the Performance report while carrying this status, you are usually looking at a data lag or a canonicalised duplicate reporting under a different URL. How To Fix “Crawled is a useful companion to this.
Why does my “not indexed” number keep growing even though traffic is fine?
Usually parameter URLs, redirects and canonicalised variants accumulating. Check the composition by reason before assuming something is broken. A growing count of correctly excluded URLs is not a problem, but it is a hint that your internal linking or faceted navigation is generating URLs it does not need to.
The takeaway
The Pages report is not a list of errors to clear. It is a record of decisions Google made about your URLs, and most of those decisions are ones you should have made yourself. Work through it in triage order, split the URLs you want indexed from the ones you do not, and apply the specific fix for each status. The sites that escape crawled currently not indexed are not the ones that click Request Indexing the most, they are the ones that publish fewer, better, well-linked pages and stop asking Google to store the rest.