Why Your Pages Are Stuck In Crawled-Currently Not Indexed & What To Do About It

Jul 20, 2026 04:30 PM - 18 hours ago 903

I person had a number of tract owners scope retired to inquire for thief pinch indexing issues lately. In astir cases, I americium uncovering that Google has classified its pages arsenic “crawled-not presently indexed” successful the page indexing study successful Google Search Console.

In almost each case, I’ve examined these pages person value issues. They are usually “commodity content” – fundamentally rehashing what galore others person already written connected a taxable without offering thing caller aliases much adjuvant than what presently exists online.

In this article, I’ll stock really I look astatine the crawled-currently not indexed study successful GSC. I’ll springiness you a instrumentality to thief you find the pages successful this study that you should analyse further. And I’ll springiness you immoderate tips for improving truthful you tin perchance recover. I must springiness adjacent warning, though. For astir sites, if you person a ample number of pages you want indexed, but they are stuck successful crawled-currently not indexed, betterment will beryllium difficult.

What Google Said About Crawled-Currently Not Indexed At The Google Search Central Event In Toronto

I attended the Google Search Central arena successful April of 2026. The organizers asked america not to property quotes straight to immoderate Googler, but they did springiness america support to stock what was said.

One presenter shared astir really Search works. He said that erstwhile Google crawls a page it fundamentally intends they download it. Then, “If we deliberation it’s useful we mightiness put it successful a database,” aliases successful different words, successful Google’s index.

Then he talked astir what kinds of things Google wants to put successful the index. He said that AI has made the period for creating things lower. If anyone tin create contented connected anything, past the type of contented Google wants to adhd to their scale is contented that offers 2 things: individual experience, and knowledge nary 1 other has.

He said that if Google has crawled your page and has decided not to scale it, location could beryllium 2 reasons:

1. There Could Be A Technical Issue

I person recovered this to beryllium rare. However, conscionable past week I reviewed a tract that had undergone a migration and each of their pages were stuck successful crawled-currently not indexed. Note: This is not the aforesaid arsenic “Discovered-not presently indexed” which intends that Google is alert of the pages, but has not yet crawled them.

My first measurement was to analyse whether Google could spot the contented connected pages. I utilized the page inspection instrumentality successful GSC by clicking the magnifying solid adjacent to the url successful the crawled-currently not indexed database and clicked, “Test Live URL” Surprisingly, erstwhile I viewed the unrecorded tested page, each it showed was a heading, a fewer boilerplate words and nary contented whatsoever.

In this case, the tract proprietor did so person a method issue. Their robots.txt had this line, Disallow: /*?*. The thought was to artifact crawling of urls pinch parameters for illustration ?replytocom aliases ?utm_source. But, their caller taxable relied connected these parameters for their CSS files and javascript truthful they were fundamentally blocking Google and each different hunt engines from seeing astir of their content.

We person since removed this artifact and very slowly, pages are starting to look backmost successful the scale again.

If your unrecorded trial shows Google tin so spot the contented connected your pages, it’s very improbable to beryllium a method rumor that is causing crawled-currently not indexed problems.

I should besides mention that immoderate pages should beryllium successful your crawled-currently not indexed database if they are not the canonical version. If you spot /feed/ pages aliases pagination aliases pages pinch url parameters, this is normal.

2. Quality

The Googler successful Toronto went connected to explicate different origin for Google to crawl a page and not scale it. He said it could beryllium because “we looked astatine it and recovered it not to beryllium good.” He said that if thousands person covered the nonstop aforesaid taxable they mightiness determine that your page is improbable to beryllium useful successful Search. It mightiness beryllium that location are different options that are much celebrated aliases of amended quality.

He besides said that sometimes Google experiments by allowing your page to beryllium indexed for a while to spot if users for illustration it, “We are experimenting pinch seeing which 1 produces happier users.” That is simply a beautiful chaotic statement!

My stake is that if you person pages that you want indexed, but Google has them successful the crawled-not presently indexed bucket, past your main rumor is related to commodity content.

Commodity Content Is The Most Likely Cause

Google talked a batch astir commodity contented astatine this event.

Commodity contented is contented that almost anyone could constitute astir a subject. It’s mostly repeating what already exists online connected different sites. Non-commodity contented brings a unsocial viewpoint aliases has contented that others deficiency aliases can’t easy replicate. It usually demonstrates first-hand knowledge aliases experience.

Take this article you are reference correct now. Anyone could usage AI to constitute a adjuvant article defining crawled-currently not indexed pages. My article, however, talks astir my acquisition arsenic a master who is paid to springiness my sentiment connected this subject. I’ve shared the real-world method illustration above, I’ve shared first-hand accusation I learned from attending a Google event, and I’m astir to stock my observations connected pages that person been deemed not worthy of indexing.

My Observations Of Pages Stuck In Crawled-Currently Not Indexed

These pages are usually not junk. They’re good, decent articles – arsenic bully arsenic the pages that Google is ranking. And that’s conscionable the point. The pages aren’t typical aliases immoderate much valuable than what presently exists.

Here is the process I usage to analyse these pages.

To find the list, click connected “Pages” nether Indexing successful GSC. Then click connected crawled-currently not indexed:

Below this, you’ll spot a database of URLs to investigate. (Below, I’ll stock much astir a instrumentality I’ve created to thief you filter this database to spot the URLs that really matter.)

I’ll find a URL successful this database that really is 1 that we want indexed.

First, I’ll hunt for immoderate queries that you would expect the page to rank for. On the SERP, location is usually an AI reply that is very helpful. Often, a personification will find the reply to their mobility there. If this is the case, past why would they want to click done to your website to publication the nonstop aforesaid thing?

I’ll grow the AI overview and past unfastened up Gemini successful the Chrome sidebar. Then I clasp down CTRL/Cmd and click connected the apical websites linked to from wrong the AIO. If you do this erstwhile you person Gemini successful the Chrome sidebar opened, you’ll find these tabs get added to your Gemini conversation.

Then I type “/” which opens up the skills I person saved astatine chrome://skills/ and take my Non-commodity check. (If you’re a member of my paid community, you tin find this afloat accomplishment here.)

This accomplishment is simply a very agelong punctual that looks astatine immoderate of the things Google tells america its algorithms purpose to reward successful its archiving connected creating adjuvant content, including, but not constricted to:

Does the contented supply original information, reporting, research, aliases analysis?
Does the contented supply insightful study aliases absorbing accusation that is beyond the obvious?
If the contented draws connected different sources, does it debar simply copying aliases rewriting those sources, and alternatively supply important further worth and originality?
Does the contented supply important worth erstwhile compared to different pages successful hunt results?

And Gemini gives maine immoderate of the reasons why the pages linked to connection worth to the reader. Note: Sometimes pages are ranking not because of their non-commodity worth but because they are an charismatic source. When you’re a known authority, you tin get distant pinch a spot much “commodity-ness.”

Now, we request to admit that Gemini does not person wrong penetration into Google’s ranking systems. It does not know why definite pages are ranking. What we are trying to study present is what types of things could beryllium helping a page beryllium worthy of presenting to searchers.

Then, I unfastened up my client’s page and punctual this, “Now analyse this page according to the aforesaid criteria.. This page is not ranking well. It is our client. Please stock wherever you deliberation it is lacking. No request to propose improvements astatine this point.”

Here is the consequence for 1 crawled-currently not indexed page I utilized this punctual on.

John Mueller And Martin Splitt Discussed Crawled-Currently Not Indexed In A Recent Podcast

As I was astir to people this, Google published a Search Off the Record Podcast connected “How to publication the Indexing Report.” There’s a batch successful here, truthful I bolded the parts that I thought were important.

This chat starts astatine 20:32 successful the video

Chapter 9: Discovered vs. Crawled Not Indexed: Is it a method aliases tract value issue?

“And also, if you adhd aliases alteration your tract aliases if your tract is very new, past you tin really besides usage this study to spot a small spot really your tract goes done the different stages, because astatine immoderate point, you’re going to spot pages successful Discovered presently not indexed. Which tells you we cognize they exist, but we haven’t really visited them. And if we haven’t visited them, we can’t put them successful the index. Crawled-currently not indexed, which intends we visited them and we didn’t put them successful the index. And that tin person each sorts of different reasons. Would you opportunity that is often aliases only sometimes a motion of a value issue?

So, it’s decidedly the lawsuit if our systems are earnestly worried astir the value of a website, that they will trim the number of pages that they index. Because if we person beardown concerns astir the wide quality, past it doesn’t make overmuch consciousness for our systems to walk a batch of clip connected the website.

So, we’ll astir apt crawl a batch less, we’ll scale a batch less, and past you’ll spot things for illustration crawled, not indexed aliases discovered, not indexed, which from our constituent of position is fundamentally our strategy saying, we cognize astir this, we looked astatine it, and erstwhile we’re happy, we will return different look and spot if we tin scale it. It’s not truthful overmuch that I would opportunity you should return these situations and effort to hole them. From a method constituent of view, it’s not that you request to hole this method rumor that Google is not indexing this page astatine the moment, but alternatively you almost request to erstwhile you admit a bigger shape for illustration this, that Google is not indexing a batch of your pages, and there’s nary method reason, you almost request to return a measurement backmost and deliberation astir the value overall.

And reasoning astir value is really challenging because a batch of times, it’s your website, and it’s your baby. And of course, it’s the champion babe ever. But taking a measurement backmost and trying to look astatine it pinch the eyes of personification who is not straight progressive pinch your website. Sometimes that opens up immoderate ideas for areas wherever you tin improve, wherever possibly if astir of your website is AI-generated and it worked for a while, it mightiness beryllium that group look astatine this AI-generated site, and they’re like, well, I tin show this is AI-generated. There’s thing unsocial aliases valuable that is disposable present for me. That’s not to opportunity that each AI-generated contented is bad, but sometimes you conscionable tally crossed websites wherever you’re like, anyone could person written this. This tells maine nothing. Yeah, that’s true. And I deliberation what makes this difficult is not only the truth that evidently the measurement you wrote it is the measurement you thought was best, and that’s why you deliberation it’s precocious quality, of course. So that’s really, really difficult to measurement retired of your ain perspective. But sometimes, there’s besides truthful overmuch different worldly that is conscionable arsenic good. So, why would we adhd it to the index.

And past that tin show you, like, possibly this contented isn’t arsenic valuable arsenic I thought it was because other group are covering the aforesaid thing. And past what’s the worth of this type of it being successful the index? Yeah that’s true. I consciousness we could person a full podcast astir quality. I deliberation possibly one different point that is worthy mentioning pinch regards to value is it’s not conscionable the text. So a batch of times people will say, well, my matter is unique, aliases my articles are good, and they’re packaged successful a page that is unspeakable to access, wherever anyone who, erstwhile they effort to load it for illustration their machine instrumentality spins up and they’re like, “Oh my gosh, I person to tally distant to make judge my machine doesn’t explode.” So possibly that’s an utmost case, but you’ve each seen these pages wherever fundamentally the matter is there, but it’s almost hidden away, hidden down ads, hidden down interstitials, hidden down different things that are moving and coming and going, possibly hidden beneath a bunch of filler content, which we sometimes see, for example, pinch recipes wherever there’s this really agelong communicative connected apical that possibly astir group don’t really attraction about. And past the look comes. These are each the kinds of things wherever the wide value is overmuch much than conscionable that portion of matter that you say, this is my main content. This is what Google should beryllium counting for my site. And from our constituent of view, we almost person to return into relationship the afloat acquisition connected a page, because that’s what users see. It’s not that users spell to a web page and move connected immoderate magic mode that conscionable pulls retired the text, but alternatively they person the afloat acquisition of this website pinch each of the 3D, 4D animations, and everything. I work together very much. Agree, ohio my God.”

How Can You Fix This Issue?

Oh boy, this is the reliable portion of this article, because successful a batch of cases, I consciousness that it is highly difficult to get pages retired of crawled-currently not indexed. I mean, if excessive ads and filler are to blame, location are evident things to amended connected there. If there’s a method issue, hole it and petition reindexing via GSC – aliases conscionable beryllium diligent and hold till Google tries to crawl your pages again.

If it’s a value issue, though, you’re apt going to person to put important effort into improving these pages.

For galore sites that I analyze, their superpower successful the past was the expertise to screen a taxable thoroughly. Lately, location is simply a inclination to not only screen a topic, but to expect each of the fan-out queries and screen those arsenic well. This was mentioned successful the Google Search Central arena a fewer times. If you are creating loads of contented based connected this method, you tally the consequence of facing a scaled contented penalty. I cannot beryllium this yet, but I fishy that the June 2026 spam update impacted a number of sites that were creating commodity contented astatine scale. If this is true, you won’t spot a manual action successful GSC. You’ll conscionable spot a driblet successful integrated postulation pinch nary explanation.

I fearfulness for a batch of SEO agencies because for many, your number 1 instrumentality successful your toolbox is contented creation. AI has made it overmuch easier to screen contented connected immoderate subject. I americium not against utilizing AI to thief pinch contented creation. But, if your SEO institution tin usage AI to create contented connected your topics, past it’s apt not original, insightful, and substantially much adjuvant than what presently exists. There are exceptions. I cognize of immoderate agencies that usage clever AI pipelines to question and reply a business, extract its applicable experience, and move that into good, original content.

Although I don’t urge utilizing AI to constitute your contented for you without immoderate quality input, I do deliberation you tin brainstorm pinch AI to thief amended it. The problem, though, is that the solutions will require effort. The connection “effort” is utilized 120 times successful Google’s Quality Rater Guidelines. You will want to find ways to tie from your acquisition to create contented that adds to the assemblage of knowledge that presently exists connected your topics.

Try this elemental prompt. Give your contented to an LLM aliases unfastened up Gemini successful the sidebar and inquire this: “Is this contented apt to beryllium considered commodity content?”

I conscionable opened up Gemini successful Google Docs and asked astir this very article you are reference now:

Next, effort this for immoderate ideas.

“Give maine 20 ideas that thief maine tie from my first-hand acquisition to make this article moreover much helpful, and substantially amended than thing other that exists connected this taxable connected the web.”

Damn, location are immoderate bully ideas successful here.

Some Tools To Help You Assess Your Crawled-Not Currently Indexed Pages

I created a mates of devices utilizing Google’s Antigravity. You tin find them astatine tools.mariehaynes.com.

There are 2 caller tools:

1. Filter your crawled-not presently indexed URLs. Export your crawled-not presently indexed URLs from GSC. If you export arsenic CSV, unfastened the zip record and find the table.csv file. You tin upload it to this tool, and it will portion retired /feed/ pages and others truthful that you tin spot and click connected the URLs that you want to investigate.

2. GSC Index Checker. You will request to log successful to your Google relationship to usage this tool, but cognize that I don’t spot immoderate of your data. It will cheque a database of URLs to spot what their indexing position is. You tin take from the astir caller pages successful your sitemap, paste a database of URLs successful manually, aliases person the instrumentality drawback your top-trafficked pages from GSC.

What you’re looking for present is whether these pages that matter to you are so indexed, aliases whether they’re stuck successful crawled-currently not indexed.