Crawl Frequency Attribute in XML Sitemaps Doesn’t Impact Crawl Rate
Google takes no notice of the crawl frequency attribute in XML sitemaps or any priority set. Only the last modification timestamp will impact crawl rate.
Google Uses Scheduler to Determine Recrawl Date
Google uses a scheduler before crawling to work out when they need to recrawl URLs. Google will increase crawl rate if it gets signals that it needs to do so e.g. updated modification date in sitemaps and internal linking (especially from the homepage).
Setting A Higher Crawl Rate Doesn’t Guarantee Google Will Crawl More
Setting your crawl rate to high means Google can crawl more but that doesn’t mean they will.
Views of Cached AMP Pages Won’t Appear in Log Files
AMP pages may make log files look unusual as Google is requesting the page to update the cache but the page will appear to receive little traffic because users see the cached version of the page.
Moving to a CDN May Temporarily Reduce Crawl Rate
If you move your site to a CDN, Google sees it as a new hosting setup and may temporarily reduce the crawl rate.
A Noindex Reduces Crawl Rate
A page with a noindex tag will be crawled less frequently.
Parameters Are Hint to Google Not to Crawl as Frequently
Parameters in Search Console aren’t a directive, more of a hint to Google not to crawl these as frequently. Blocking URLs in Robots.txt is strong directive to say Google should not be looking at these pages e.g. if Google’s crawls are crashing your site.
Submit Sitemap With Updated Last Modification Date For Faster Crawling of Updated Pages
Submit a sitemap file with an update last modification date to speed up the process of crawling and indexing of pages that have been changed.
Google Recommends Keeping Redirects For At Least One Year
Google recommends keeping redirects for one year minimum. This is because URLs will, in the worst case scenario, be crawled by Googlebot every six months, so a having the redirect in place for a year would give Google the chance to see that redirect at least twice.