Disallowed Pages

What does this report contain?

All URLs included in the crawl which were disallowed in the robots.txt on the live site, or from a custom robots.txt file in Advanced Settings.

How can this report be used?

These URLs will not be crawled by Google, unless they are redirected to from other URLs which can be crawled. They will be indexed in Google, but won't show a title or description and are unlikely to rank well, so they should be reviewed to ensure they are low value.

Categories:

Uncrawled Urls, Indexation

API Report Code:

disallowed_pages