HTTP & Robots Cheatsheet

Sam Marsden
Sam Marsden

On 9th December 2015 • 0 min read

This cheatsheet covers every status code and robots directive, and should get you thinking about the impact of using each one.

Nothing is as simple as it seems though, so anything coloured in orange is conditional, or may only be true under special circumstances.

So let us know if you disagree with anything.

Also available in PDF.


Sam Marsden
Sam Marsden

Sam Marsden is DeepCrawl's SEO & Content Manager. Sam speaks regularly at marketing conferences, like SMX and BrightonSEO, and is a contributor to industry publications such as Search Engine Journal and State of Digital.



Get the knowledge and inspiration you need to build a profitable business - straight to your inbox.

Subscribe today