By Shad Super in Questions —

How Can I Use the URL Inspection Tool to Diagnose Why Specific Pages Are "Crawled - Currently Not Indexed" and Prioritize Them for Indexing?

Summary

The URL Inspection Tool in Google Search Console allows you to diagnose why specific pages are "Crawled - currently not indexed" and prioritize them for indexing. This process involves analyzing Google's crawl and indexing process, identifying probable causes, and implementing possible remedies. Relevant strategies might include ensuring the site’s robots.txt file allows Googlebot to access your page, checking for noindex directives, confirming that Google can access the page, and enhancing your page's content quality and relevance.

Understanding Why Pages are Not Indexed

Accessing the URL Inspection Tool

Access the URL Inspection Tool in Google Search Console by entering the complete URL of a specific page into the search bar at the top of the page. The "Coverage" section provides information about the page's index status and information about why Google might not have indexed the page [Google Crawlers Overview, 2021].

Interpreting "Crawled - Currently Not Indexed"

A message of "Crawled - currently not indexed" means Google has crawled the page, but it's not indexed, possibly due to technical reasons, "noindex" directives, relevancy, or quality issues [URL Inspection Tool Guide, 2021].

Diagnosing Indexing Issues

Check Robots.txt File

Ensure your site’s robots.txt file isn't blocking Googlebot from accessing your page as it could prevent Google from indexing those pages. Test the URLs using the robots.txt testing tool in Google Search Console [Robots.txt Specification, 2022].

Noindex Directives

Check for any "noindex" meta tags on your page, which tell Google not to index the page. If a page isn't intended to be de-indexed, you must remove these directives [Robots meta tag and X-Robots-Tag specification, 2022].

Accessibility Check

Google must be able to access your page to index it. Check and ensure that your page is properly connected to your site through a linked network of pages, or URL structure, and can be accessed without the need for form submission, cookies, or other interactive features [Google Search Guidelines, 2022].

Improving Content Quality and Relevance

If all technical checklists pass, Google might not index your pages due to perceived low-quality or non-relevant content. Review and enhance the quality, relevance, and uniqueness of the content to meet Google Search's content quality guidelines [Quality Guidelines, 2022].

Provide Valuable Content

Your page must offer valuable information to users to grab Google's attention. Prioritize creating content with substance, relevance, and uniqueness, while making sure to optimize titles and descriptions to accurately represent the content [Content and SEO, 2022].

Conclusion

By understanding and utilizing the URL Inspection Tool in Google Search Console, you can diagnose why certain pages are "Crawled - currently not indexed" and prioritize them for indexing. It involves ensuring your site is accessible to Google, there are no indexing directives that may block Google, and your content is high-quality, relevant, and valuable to the desired audience.

References