Noindex
A rel=”noindex” directive is used to instruct search engines not to include a page within their index, to prevent it from appearing within search results. Our SEO Office Hours Notes below explain the use of this directive, along with further advice compiled from Google’s Office Hours sessions and real-world examples.
For more on noindex directives, check out our article: Noindex, Nofollow & Disallow.
Submit Subdirectories to URL Removal Tool to Get Around Individual URL Limits
The URL Removal Tool limits the number of individual URLs that can be submitted to be removed per day. To get around this, you can submit subdirectories to get entire sections of content removed from Google’s index.
Number of Noindexed Pages Has No Effect on Rankings or Site Quality
Having a lot of noindexed pages doesn’t affect rankings or how Google perceives a site’s quality. For example, many sites need to noindex private content that requires a user to log in to access.
Many-to-one Redirects & Noindexed Pages Are Sometimes Treated as Soft 404s
Noindexed pages and too many pages that redirect to one URL can both be treated as soft 404 errors by Google. Having soft 404s doesn’t impact the perceived quality of your website, but these pages won’t be crawled as frequently or indexed at all.
Avoid Using Google Tag Manager to Implement Critical Tags Like Noindex
John suggests that search engines other than Google may struggle to process GTM tags. Also, because tags are powered by JavaScript, they will only be rendered and applied to a page a few days or weeks after the initial HTML page is indexed.
Google May Take Longer to Process Redirects For Noindexed Pages
Google crawls noindexed pages less frequently. If a redirect is set up for a noindexed page, Google may take longer to process this because it is being crawled less frequently.
Noindexing Some Pages with Hreflang Tags Won’t Affect Whole Configuration
It’s fine to noindex some pages within a set which have hreflang tags implemented. The noindexed URL will simply be removed from the set that Google uses, without affecting the other pages.
AMPs Which Canonicalise to Noindexed Pages Won’t be Valid
If a page is noindexed which is connected to an AMP equivalent, then the rel=amphtml link will be dropped too and the AMP won’t be valid. If the AMP is noindexed then it will be removed from the index and the traditional page will be kept.
Don’t Rely on Unsupported Robots Directives in Robots.txt Being Respected By Google
Don’t rely on noindex directives in robots.txt as they are aren’t officially supported by Google. John says it’s fine to use robots directives in robots.txt, but make sure you have a backup in case they don’t work.
A Sitemap is the Best Way for Google to Quickly Process Noindex at Scale
Make sure the pages you’ve added a noindex tag to are included in a sitemap file with the last modified date to ensure Google picks these up as quickly as possible. Make sure last modified dates are realistic and aren’t the same for every page as this looks artificial to Google.
Google Eventually Treats Noindexed Pages as Soft 404s
Google treats noindexed pages as soft 404s after a period of time, as the page is ignored and essentially doesn’t exist in their eyes.