Segmentation in Deepcrawl

Helen Cherry

On 22nd October 2021 • 9 min read

Helping our customers improve their SEO and earn impactful gains in their search results is at the heart of our business at Deepcrawl — and we’re constantly working on creating even faster, easier ways for our customers to execute their SEO strategies at scale and improve their websites’ performance. To that end, we have been making major improvements in our new application, with segmentation as a foundational element of the technology stack.

What is segmentation?

Segmentation allows you to easily conduct analyses on different page types across your website and efficiently identify SEO issues and opportunities.

With segmentation you can:

For example, you can create segments for different product categories on your website or for different country/language versions.


Getting started

Access to the new application is only available for enterprise clients. To find out more, please speak to one of our account executives.

There are 3 key steps to using segmentation:

  1. Create your segments
  2. Run a crawl
  3. View the reports


How do I access segmentation in Deepcrawl?

You can access segmentation from the project dashboard or any of your Deepcrawl reports.

How to access segmentation features from the Deepcrawl dashboard

To switch to the updated version of the Deepcrawl app (and access our segmentation features), look at the example below. Within the report header, you will see a “New App” link. Click on it and you will be switched to our new app.

How to switch to the new version of the Deepcrawl SEO app

When you switch to the new Deepcrawl app, you will see our new report template and design. The first time you navigate here, the charts will be empty – to populate them you will need to create your segments.

Example of the new Deepcrawl SEO report designs



How do I create segments?

Once you’re in the new app, you’ll simply need to select “new segment” from the drop-down menu in order to create and configure the desired segments for your project.

How to add a new segment to your Deepcrawl SEO reports


Creating a segment:

  1. A segment must have:
    1. A name 
    2. At least 1 rule defined
  2. You can create segments with most of the filters we have on our reports
  3. If you choose regex validation, please see our hints and tips section
  4. Once name and rules have been filled out, click “Create Segment”
  5. The new segment will be added to the segment manager

Creating and managing new segments in the Deepcrawl app for enterprise companies

Crawling & segmentation

Once you are happy with the segments you have created, then you must run a new crawl. To do this, just navigate back to step 4 in Project Settings and run a new crawl from there:

Run a crawl with your new segments added by navigating to Deepcrawl's Project Settings tab

Notes on generating a segment:

1. A new crawl must be run for a segment to be generated.

2. You can also now preview a segment before creating it:

How to preview a segment before crawling your website with Deepcrawl


3. A segment begins to generate after a new crawl has finished.

4. You can create segments during the “crawling” state and they will also generate.

5. If you create a segment during the “finalizing” state, however, you will need to run a new crawl for a segment to generate.

(An example of a crawl during the “finalizing” stage is below — create your segments before this stage, or before running a crawl.)

Example of a Deepcrawl crawl during the finalization stage — create segments before this stage


6. You can create segment rules not only by URLs ,but also by any other metric available in Deepcrawl reports:

Creating segment rules by any Deepcrawl metric
7. You can also use “AND” / “OR” statements in your segments:
How to use and - or rules in your Deepcrawl segmentation settings

8. Now both “Issues” and “Changes” reports can also be segmented in Deepcrawl:

How to add segmentation to your Issues and Changes reports within Deepcrawl's SEO SaaS platform

Viewing reports

After you have run your crawl and your segments have been successfully generated, you can navigate back to “Reports” to view. You will initially be taken to reports that display the following data:

Crawl reports in the Deepcrawl Dashboard after creating segments on the platform
You can then select a specific segment by clicking the “view segment” radial and selecting which segment you wish to analyze. Alternatively, you can click on any data bar within the charts.

Analyze your SEO crawl segments from the graphs

When a segment is applied, the data in the report will be updated based on the rules in the segment.

The applied segment is also persistent across the new core app, which means that you can navigate to another report and the segment is still applied.

A selected segment is applied across all reports in the Deepcrawl app


If the segment has been generated in past crawls then it will even persist when you switch to an unarchived historic crawl:

How to see a segment in a historic crawl



You can also apply local filters on top of your segments:

Applying local filters on top of a segment in Deepcrawl

Applying additional local filters on top of a Deepcrawl segmentation for additional technical SEO insights

Exporting segment data

When a segment is applied, the data can be exported into a CSV file. Simply go to the “Export data” button and select “Generate CSV [Segment]”.

Exporting a CSV with segmentation data


The CSV file will generate and then be downloaded to your browser.

Finding the exported CSV


As well as downloading the segment report, you can also download the segment report with filters added:

Exporting a filtered CSV with segmentation on Deepcrawl
If you want to delete any downloaded files,  simply click on the bin icon to remove them from the downloads dropdown.




Top Segments graph:
Viewing the top segments graph in the new Deepcrawl app

The Top Segments graph displays the top 5 segments that have the most URLs in a report. It can be used to quickly highlight problem areas of a website in any report.

The data points are clickable so you can simply click on any segment in the graph and it will take you to that segment in the report (which will persist when you move to another report in the new core app).


Segment Report Graph
Viewing the Segment Report graph in the new Deepcrawl app
The segment report chart compares the currently selected segment URLs against all other URLs in this report. It allows customers to see how big the URLs are within the context of the whole report.

If you have only created a handful of segments and these segments are quite small in the context of the rest of the pages in a report, then it indicates that more segments need to be created.


Trend Chart

Viewing the Trend Chart in Deepcrawl's platform

The trend chart maps out the data trends over the last 30 crawls for any segment report. It can help you track improvements or issues for particular areas of the website.

For example, the number of pages over the medium fetch time being crawled spiked in the last crawl in the /2020/ area of a large publisher website.

Viewing the Medium Fetch Time over time chart in the new app

Additional Hints and Tips for Segmentation in Deepcrawl

Regex validation

If you choose “Matches (Regex)” and “Doesn’t Match (Regex)” rules, please enter Lucene valid regular expressions. Use Rubular to help write valid regular expressions. Forward slashes do not need to be escaped.


Managing and copying segments

  1. You can delete a segment, a tooltip will always ask if you want to delete to ensure you don’t accidentally delete a segment
  2. You can view rules for each segment created
  3. You can see when a segment was generated
  4. You can bulk copy a group of segments to another project in your account.




Choose a better way to grow

With tools that will help you realize your website’s true potential, and support to help you get there, growing your enterprise business online has never been so simple.

Book a Demo