Sitemaps

A sitemap is a list of all of the live URLs which exist on a site and is used to inform search engine crawlers of the most important pages and therefore which ones should be crawled and indexed. There are several things to consider when creating sitemaps, as well as understanding how search engines view them. We cover a range of these topics within our Hangout Notes, along with best practice recommendations and advice from Google.

No Good Solution for Reactivating Pages

July 1, 2016 Source

If you have pages which expire but are reactivated after a period of time, there isn’t really a good solution, but you can use a Sitemap to tell Google about URLs which are now active, and use the unavailable-after meta tag.


Mobile Sites Don’t Need Sitemaps

May 17, 2016 Source

Separate mobile sites should be canonicalising to the desktop page, so you don’t need to submit them to Google via a Sitemap, but it’s still worth adding to Search Console.


HTML sitemaps help indexing and crawling

February 26, 2016 Source

If you have a complicated website, providing a mapping of your category pages can help Google to find pages and understand the structure of a website.


Cross Domain Sitemaps Will Be Crawled If Present in Robots.txt

February 23, 2016 Source

Google will use Sitemaps hosted on an external domain if they are referenced in the robots.txt.


RSS + PubSubHubbub is better than XML sitemaps

September 11, 2015 Source

John recommends using RSS with PubSubHubbub as the fastest way to get new content indexed.


Submit Canonical URLs in Sitemaps and Other References

April 24, 2015 Source

Use consistent canonical URLs in Sitemaps and other internal references.


Use Sitemaps for Redirect Discovery

April 24, 2015 Source

If you want Google to see your redirected URLs, such as after a URL change, it’s OK to submit the old URLs in a Sitemap to help Google recrawl them more quickly


Submit Expired Pages in XML Sitemaps

December 23, 2014 Source

You can submit an XML Sitemap with expired pages to help get them removed from the index more quickly. It’s best to put them into a separate sitemap so you can see them separately to other indexable URLs.


XML Sitemaps Are Most Beneficial for Large Sites

December 23, 2014 Source

XML Sitemaps are not an essential requirement, and are most beneficial for sites with changing or a large turnover of content.


Related Topics

Crawling Indexing Crawl Budget Crawl Errors Crawl Rate Disallow Last Modified Nofollow Noindex RSS Canonicalization Fetch and Render