Why doesn’t Google index my content properly?

December 11, 2018

Before Google can rank your content, it needs to discover it, be permitted and able to evaluate it, and index it. If any of these processes go wrong, you might find that your pages don’t show up in the search results.

Most of the time, you can rely on Google to correctly index your content, all by itself. After all, this process is one of the foundational parts of what Google is, and does.

Simply putting your content online isn’t always enough, however.

If you have technical issues, low-quality content, or incorrect indexing controls, you might trip up during those processes of discovery, evaluation and indexing might go wrong.

Discovery

In order to index a page, Google has to be able to find it. That means that somewhere has to link to it – whether that’s from other indexed pages in the same site, or from other sites.

Depending on the relevance and quality of the places it’s linked from, it might take a little time for Google to schedule following those links and finding your pages.

That also means that the page can’t be ‘hidden’ – which, for example, might mean content being password protected, blocked via robots.txt, or only available to users in certain countries.

Evaluation

When Google has discovered the page, it will digest the content (including the HTML code and related assets) to assess the quality and relevance.

During this process, there are a number of things which can result in Google choosing not to index a page. They include:

  • When it determines that the content of the page is ‘low quality’. E.g., if there’s a very low word count, or if the content is a close/direct duplicate of another page. Particular ‘over-optimized’ or ‘spammy’ pages may also be ignored.
  • When it discovers specific indexing instructions on the page (such as a meta robots tag, or a canonical URL tag pointing at a different page). Google will make a judgment call in cases like this whether it should honor the instructions, but chances are, it’ll choose not to include the page.
  • When it can’t see/access the content. For websites which rely heavily on JavaScript, or those which include content in complex or non-standard ways, Google might not be able to consume the page content. It may be that, as far as they’re concerned, it’s an empty (or low quality) page.
  • When it has to process heavy JavaScript, Google might schedule a ‘follow-up’ crawl to dig deeper, before deciding what/whether to index or not. The time this takes can vary considerably, based on Google’s resourcing and their prioritization of your pages.

Indexing

If you’ve passed all of those tests, then your content should be successfully indexed and should turn up when you search for it.

HINT: try doing a ‘site’ search on Google (e.g., site:https://www.example.com/example-page/) to see if a specific URL has been included in the index).

Bear in mind that, once a page is in the index, that doesn’t mean it’ll stay there forever! Google repeatedly crawls and re-evaluates content – so if your quality drops, or if you accidentally prevent Google from evaluating the content, then your page might get dropped out of the index.

Any questions? Let us know in the comments!

Read more: What is crawlability? »

Share this article:
 




eHost-square-ad Theme Builder Layout

We’re listening.

Have something to say about this article? Share it with us on Facebook, Twitter or LinkedIn:

SHARE IT HERE:

Related Posts

gutenberg-one-year-later Theme Builder Layout

Gutenberg: One Year Later

As we quickly head into the final weeks of 2019, we also pass the first anniversary of WordPress 5.0 and, subsequently, Gutenberg coming headlong into our lives. Love it or hate it, Gutenberg is here to stay. If you had asked my thoughts on it last December, I would...

initial-documentation-for-block-based-wordpress-themes-proposed Theme Builder Layout

Initial Documentation for Block-Based WordPress Themes Proposed

In a pull request on the Gutenberg repository yesterday, Riad Benguella created an initial document that outlines how block-based WordPress themes might work. While the document is merely a starting point for the conversation, it is a set of ideas that will likely...

Get ALL Your SEO, WordPress & Divi News

Join Our Daily Roundup

SEO News and More

SEO News and More

Subscribe ToThe Weekly SEO Trade News Updates

Get the latest SEO, SEM and SMM marketing intel, tips and tricks from one of the best SEO Gurus online. 

Every Tuesday morning we send out an aggregated email listing all new posts on SEO Trade News.

Excellent! Now check your email to confirm your subscription.