Robots Control

Best Practice

8th July 2019 • 6 min read

Robots.txt Noindex Update: Everything SEOs Need to Know

Update: As of 1st September 2019, Google will be retiring all code that handles unsupported and unpublished rules in robots.txt…

Best Practice

24th January 2019 • 6 min read

Common Robots.txt Mistakes and How to Avoid Them

Robots.txt is a critical tool in an SEO’s arsenal, which is used to establish rules that instruct crawlers and robots…

Best Practice

3rd November 2015 • 13 min read

Disallow and Google: An Intermediate SEO Guide

Following on from our beginner’s guide to implementing noindex, disallow and nofollow directives, we’re now taking a look at some…

Best Practice

28th September 2015 • 9 min read

Noindex and Google: A Guide for Intermediate SEOs

Following on from our beginner’s guide to implementing noindex, disallow and nofollow directives, we’re delving a bit further into the…

Best Practice

17th September 2015 • 16 min read

Noindex, Nofollow & Disallow

The three words above might sound like SEO gobbledegook, but they’re words worth knowing, since understanding how to use them…

Best Practice

26th May 2015 • 12 min read

How to Audit URLs Being Blocked By Robots.txt

When auditing websites that have experienced a drop in rankings and organic search traffic, it’s important to have a good…

Best Practice

17th July 2014 • 8 min read

Managing robots.txt Changes

Writing and making changes to a robots.txt file can make even the most hardened SEOs a little bit nervous. Just…

Choose a better way to grow

With tools that will help you realize your website’s true potential, and support to help you get there, growing your enterprise business online has never been so simple.

Book a Demo