Splunk Integration

Our Splunk integration allows us to automatically get log file data every time we crawl a website, meaning that data is always up to date, and there is no need to manually upload data.

Read more

Majestic Backlink Data

We’re proud to announce DeepCrawl’s latest partner integration and another fantastic addition to your Search Universe: Backlink data from Majestic.

Read more

Supported Upload Files

DeepCrawl supports the uploading of a wide range of file types to crawl sources. This page runs through the valid file types for each source, and where to find them.

Read more

Limiting the Size and Depth of a Crawl

This guide explains how you can restrict the overall size and depth of a crawl before you start or during your crawl. This is useful to prevent a lot of URL credits being used unintentionally, or to run a discovery crawl, when you first start to crawl a website and don’t yet know the optimal settings.

Read more

Four Awesome Things You Can Do With Regex

Many of DeepCrawl’s features are centred around identifying and monitoring issues with site architecture. But the tool can also be used creatively to improve user experience, gather data about the structure of your site, and even make non-technical tasks such as seeking out text on your site easier and more reliable. We thought we’d share four examples of how using…

Read more