Noindex

A rel=”noindex” directive is used to instruct search engines not to include a page within their index, to prevent it from appearing within search results. Our Hangout Notes explain the use of this directive, along with further advice from Google and real world examples.

Google May Take Longer to Process Redirects For Noindexed Pages

July 27, 2018 Source

Google crawls noindexed pages less frequently. If a redirect is set up for a noindexed page, Google may take longer to process this because it is being crawled less frequently.


Noindexing Some Pages with Hreflang Tags Won’t Affect Whole Configuration

July 24, 2018 Source

It’s fine to noindex some pages within a set which have hreflang tags implemented. The noindexed URL will simply be removed from the set that Google uses, without affecting the other pages.


AMPs Which Canonicalise to Noindexed Pages Won’t be Valid

July 24, 2018 Source

If a page is noindexed which is connected to an AMP equivalent, then the rel=amphtml link will be dropped too and the AMP won’t be valid. If the AMP is noindexed then it will be removed from the index and the traditional page will be kept.


Don’t Rely on Unsupported Robots Directives in Robots.txt Being Respected By Google

July 13, 2018 Source

Don’t rely on noindex directives in robots.txt as they are aren’t officially supported by Google. John says it’s fine to use robots directives in robots.txt, but make sure you have a backup in case they don’t work.


A Sitemap is the Best Way for Google to Quickly Process Noindex at Scale

July 10, 2018 Source

Make sure the pages you’ve added a noindex tag to are included in a sitemap file with the last modified date to ensure Google picks these up as quickly as possible. Make sure last modified dates are realistic and aren’t the same for every page as this looks artificial to Google.


Google Eventually Treats Noindexed Pages as Soft 404s

June 15, 2018 Source

Google treats noindexed pages as soft 404s after a period of time, as the page is ignored and essentially doesn’t exist in their eyes.


Videos Blocking Googlebot May Still be Crawled and Indexed

June 12, 2018 Source

Blocking Googlebot from crawling a video may still result in a video snippet appearing in search if the video file is embedded from a different location, if some Google datacentres haven’t yet seen the updated version or if the video URL has parameters attached.


Noindex & 410 Pages Are Removed Faster Than 404

June 1, 2018 Source

Noindex and 410 remove pages from Google’s index at about the same speed, and both are slightly quicker than using a 404.


Adding Noindex with JavaScript Isn’t Recommended, Use Static HTML

May 29, 2018 Source

If you add a noindex tag using JavaScript then its effects will be delayed as Google will only pick it up in the second wave of rendering and indexing. Google can read the tag after rendering but static HTML is recommended instead.


Related Topics

Crawling Indexing Crawl Budget Crawl Errors Crawl Rate Disallow Sitemaps Last Modified Nofollow RSS Canonicalization Fetch and Render