A sitemap is a list of all of the live URLs which exist on a site and is used to inform search engine crawlers of the most important pages and therefore which ones should be crawled and indexed. There are several things to consider when creating sitemaps, as well as understanding how search engines view them. We cover a range of these topics within our Hangout Notes, along with best practice recommendations and advice from Google.

New Search Console Will Show More Sitemap Data

December 12, 2017 Source

The new Search Console will show more detailed information regarding sitemaps and more detail per sitemap file.

URLs in Sitemaps Are Not Guaranteed to be Indexed

November 14, 2017 Source

Google may choose not to index URLs in sitemaps that are very similar to ones already being indexed and if they differ to the ones linked within the site (e.g. trailing slash/non-trailing slash).

Mixed Migrations May Cause Google to Index HTTP or HTTPS URLs

November 3, 2017 Source

Forgetting to update your sitemap files following a HTTPS migration could cause some pages to be indexed with the HTTP URL and some HTTPS.

Video Sitemaps Can Specify Countries Where Content is Available

October 31, 2017 Source

With a video sitemap you can define which countries your content is available, which is used for video search results.

Low Proportion of Indexed Pages Points to Technical Issue

September 22, 2017 Source

If a site has a low proportion of indexed pages, this usually points to a technical issue than a quality issue. Compare the site map index counts and index status report for differences. Try splitting up sitemap file , checking indexed pages using info: query, that rel canonicals match those in sitemap file, hreflang and internal linking. Also, uppercase, lowercase, trailing slashes all matter. Then check crawl stats to get idea of crawl rate and if it’s reasonable.

GSC Sitemaps Report Can Take Couple of Days to Update

September 8, 2017 Source

Sitemaps report in GSC can take a couple of days to update after changes have been made to the sitemap and may explain why non-existent errors are reported.

Google Validates Sitemap Files Immediately After Submission

September 5, 2017 Source

The 50k URL limit for sitemaps is based on the number of entries or elements in the sitemap file (including alternate linked URLs) and this is validated immediately after they are submitted. So if there are too many URLs in the sitemap file, you will be made aware of that straight away.

Ensure Separate Sitemap Files Don’t Contain URL Overlap

August 11, 2017 Source

Having separate dynamic and static sitemap files is fine, as long as there is no URL overlap.

Submit Sitemap With Updated Last Modification Date For Faster Crawling of Updated Pages

August 11, 2017 Source

Submit a sitemap file with an update last modification date to speed up the process of crawling and indexing of pages that have been changed.

Related Topics

Crawling Indexing Crawl Budget Crawl Errors Crawl Rate Disallow Last Modified Nofollow Noindex RSS Canonicalization Fetch and Render