Technical SEO indexing problems: a diagnosis and fix flow
One flow for answering why a page is not indexed: reading Search Console statuses, verifying with URL Inspection, finding the root cause, proving the fix.
By Roozbeh Nazari · CEO
"Our page isn't showing on Google" is the most frequently heard and least informative complaint in technical SEO. The page may not have been crawled, may have been crawled but not indexed, may be indexed but overshadowed by another URL, or may be indexed but not visible for the query being searched. The fixes for these four situations are entirely different, and the first job is to work out which one you are in. This article proposes a fixed diagnosis flow for that: the same order, the same tools, every time.
The flow is the written version of the sequence we apply to every indexing complaint in our technical SEO work. For the broader crawl-side checklist, see the 20-item production checklist.
Step 1: turn the complaint into a status
Search Console's Page indexing report places every URL in a status that Google itself names. These names are the language of diagnosis, and the first step of the flow is to translate the complaint into one of them. The statuses in the report fall roughly into four clusters.
- Access barriers: server error (5xx), redirect error, blocked by robots.txt, 401 and 403 blocks, not found (404), soft 404.
- Explicit instruction: URL marked noindex.
- Google's choice: crawled, currently not indexed; discovered, currently not indexed.
- Duplicate handling: alternate page with proper canonical tag, duplicate without user-selected canonical, duplicate where Google chose a different canonical than the user, page with redirect.
Each cluster means something different. An access barrier is a technical fault, an explicit instruction is a decision you made, Google's choice is a quality or capacity signal, and duplicate handling is an architecture problem. Starting to fix before finding the name in the report means guessing which cluster you are in.
Step 2: verify on a single URL
The report gives an aggregate view and updates with a delay. The diagnosis has to be verified on a single URL with the URL Inspection tool. The tool tells you two things: what Google's indexed version knows about this page, and whether the page is indexable today when a live test is run. The difference between the two shows whether the problem is still ongoing; if the indexed version is faulty but the live test is clean, the fix has been made and a recrawl is pending.
The live test also has an output that tends to be ignored: the HTML as Google rendered it. The page you see in the browser and the page Google renders are not necessarily the same; if the title, canonical and main content are missing from the rendered HTML, the cause of that gap has to be found before moving on with the rest of the diagnosis. Most of the time the cause is a component loaded late on the client side, or a layer that answers bots differently.
Two fields matter in the live test: whether crawling is allowed, and the page's selected canonical. On multilingual sites the second field is full of surprises; cases where Google picks the English version as canonical for a Turkish page usually come from hreflang and canonical not agreeing with each other.
Step 3: look for the root cause by cluster
In the access barrier cluster the root cause is almost always infrastructure. 5xx errors sit in the server or application layer; redirect errors in a chain or loop; a robots.txt block in the file itself, sometimes in the wrong environment's file going live with a deployment. A soft 404, meaning the page returns 200 but its content is empty or says "not found", often comes from client-side loaded content arriving empty for Googlebot.
Then there are servers that treat bots and users differently. Sites that auto-redirect by the user's language redirect Googlebot too, and some language versions are never crawled. We lived through this case on our own site and described it, with the test method, in our locale auto-redirect article; at this step of the flow, comparing the response Googlebot gets with the response a user gets is mandatory.
In the explicit instruction cluster the root cause is a noindex tag or header. The question here is not "where is the tag" but "who put the tag there, and why". It may be a template moved from staging to production, a page-level setting in the CMS, or an SEO plugin's default. If you do not find the source before removing the tag, it comes back with the next deployment.
The Google's choice cluster is the hardest, because technically everything looks correct. "Discovered, currently not indexed" is usually crawl capacity or weak internal linking; "crawled, currently not indexed" is usually the page not being found sufficiently distinct compared with the rest of the site. The fix here is not technical: linking to the page from within the site, merging similar pages, or making the content genuinely different.
In the duplicate handling cluster the root cause is in the architecture: parameterised URLs, trailing slash differences, http and https versions, the same content published under two language versions without a canonical. Google choosing a different canonical than the user is not Google's mistake but two signals contradicting each other; redirects, canonical and hreflang need to point at the same URL.
Step 4: prove the fix, do not assume it
After a fix, three pieces of evidence are needed. First, the live test in URL Inspection coming back clean. Second, Googlebot actually re-fetching the page after the fix, seen in the server log; without that the index status does not change. Third, the status changing in the report; this third piece can take days or weeks, and the only tool for shortening it is requesting indexing from URL Inspection.
Requesting indexing should not be seen as a bulk repair tool. It works for individual important pages; for hundreds of pages, what needs doing is keeping the sitemap current and making sure internal links actually reach those pages. Google's robots.txt documentation states a frequently confused point plainly: robots.txt is not a mechanism for keeping a page out of Google; that takes noindex or password protection. This holds in the reverse direction too: putting noindex on a page blocked by robots.txt has no effect, because Google cannot crawl the page to see the tag.
Why keep the flow fixed
Applying these four steps in the same order every time is less about diagnosis speed than about preventing the wrong fix. The most common mistake we see is changing the canonical for a page in "crawled, currently not indexed", or rewriting content for "discovered, currently not indexed"; both come from misreading the cluster and cost weeks. Because the flow ties every fix to a status name and a piece of evidence, the record of what was done and why builds itself.
Conclusion
An indexing problem is not one problem but a situation that falls into one of four clusters. The diagnosis flow starts by translating the complaint into the status Search Console names, is verified with a live test on a single URL, looks for the root cause by cluster and closes the fix with three pieces of evidence. Teams that apply this order without breaking it fix the same page once rather than three times.