In order for web pages to be included within search results, they must be in Google’s index. Search engine indexing is a complex topic and is dependent on a number of different factors. Our SEO Office Hours Notes on indexing cover a range of best practices and compile indexability advice Google has released in their Office Hours sessions to help ensure your website’s important pages are indexed by search engines.

Duplicate Content Snippets are Credited to the Most Relevant Version

March 8, 2016 Source

If you have pages with duplicated snippets of content, such as product descriptions, Google will show the page which it thinks is the most relevant version.

HTML sitemaps help indexing and crawling

February 26, 2016 Source

If you have a complicated website, providing a mapping of your category pages can help Google to find pages and understand the structure of a website.

Google Establishes Original Content Sources

February 23, 2016 Source

If multiple sites are using the same affiliate/product content, Google will try to establish the original source, and allow a few other version to rank. They will be treated as separate sites which can rank independently for different queries. The suggestion is that this is fine provided you include some unique content. Later he also suggests this for pages with embedded videos.

Descriptions Rewritten by Google Don’t Affect Rankings

February 23, 2016 Source

If Google is replacing your description tags, it won’t affect your rankings as descriptions are not used, but it says Google doesn’t think your descriptions are relevant for the page.The reasons may be; ontains spammy keywords, too vague, too promotional, doesn’t match the content on the page

302s Are Indexed, 301s Aren’t

February 12, 2016 Source

Google always use the content from the final destination page, but will display either the redirecting URL for a 302 redirect, or the the final destination for a 301 redirect. They ignore any content on the redirecting URL.

Valuable Site Search Results Pages Can be Indexed

February 6, 2016 Source

Google will crawl site’s own search results pages if they are linked, and may index them if they show content which Google thinks is valuable.

Disallowed URLs can be Indexed

January 29, 2016 Source

Even if a URL is disallowed it can still show up in the index.

Google Indexes Text in Form Fields

December 1, 2015 Source

Google does look at text in form field values, such as drop downs, but not HTML comments.

Search Console Indexed Pages Count Includes Duplicates

December 1, 2015 Source

The Index Count in Search Console may include duplicates which may be filtered out of results, and suggests Sitemaps containing your unique URLs as the best method for measuring indexing.

Related Topics

Crawling Crawl Budget Crawl Errors Crawl Rate Disallow Directives in Robots.txt Sitemaps Last Modified Nofollow Noindex RSS Canonicalization Fetch and Render