Segmentation

Helping our customers to improve their SEO is at the heart of our business and finding easier and faster ways to do this has been our focus for the past few months. To that end we have begun the process of completely rebuilding our application with segmentation as the foundation of the technology stack.

What is segmentation?

Segmentation allows you to easily conduct analysis on different page types across your website, and efficiently identify issues or opportunities.

With segmentation you can:

• Create custom segments based on DeepCrawl’s 250+ metrics
• Identify SEO issues quickly by only focusing on priority areas of your business
• Discover and monitor SEO trends in priority areas of your website.
• Export segment data into Google Data Studio using the DeepCrawl Connector to create customized dashboards

Getting started

Access to the new application is only available for enterprise clients – to find out more please speak to one of our account executives

There are 3 key steps to using segmentation:

1. Create your segments
2. Run a crawl
3. View the reports

How do I access segmentation?

In an existing project go to the dashboard and then navigate to any report, for example Primary Pages, Duplicate Pages, Non-indexable pages, etc clicking on the respective data label in the chart.

In this example (Canonicalized and Noindexed Pages) within the report header you will see a Go to Segmentation View link. Click on it and you will be switched to our new app.

When you switch to the new app you will see our new report template and design. The first time you navigate here the charts will be empty – to populate them you will need to create your segments.

How do I create segments?

Once in the new app to create and configure your segments for your project, click on the My Segments button.

Creating a segment

1. A segment must have
   1. A name, and
   2. At least 1 rule defined
2. Right now you can only segment on URL but we plan to add more metrics very soon
3. We only support AND rules for segment rules but more complex rules will be available in the future
4. If you choose regex validation please see our hints and tips section
5. Once name and rules have been filled out hit Create Segment
6. The segment will be added to the segment manager

Crawling & segmentation

Once you are happy with the segments you have created then you must run a new crawl. To do this navigate back to step 4 in project settings and run a new crawl from there.

Notes on generating a segment:

1. A new crawl must be run for a segment to be generated
2. A segment begins to generate after a new crawl has finished
3. You can create segments during the “crawling” state and they will generate
4. If you create a segment during the “finalizing” state, however, you will need to run a new crawl for a segment to generate.

Viewing reports

After you have run your crawl and segments have been successfully generated you can navigate back to the reports to view. You will initially be taken to reports that display the following data:

• Top segments in a report
• Trend graph (max 30 crawls).

You can then select a specific segment by clicking the view segment radial and selecting which segment you wish to analyze. Alternatively you can click on any data bar within the charts.

When a segment is applied the data in the report will be updated based on the rules in the segment. The segment is also persistent across the new core app, which means that you can navigate to another report and the segment is still applied.

If the segment has been generated in past crawls then it will even persist when you switch to an unarchived historic crawl.

You can also apply local filters on top of your segments:

Exporting segment data

When a segment is applied the data can be exported into a CSV file. Simply, go to the Export data button and select “Generate CSV [Segment]”.

The CSV file will generate and then be downloaded to your browser.

In the new app you can also cancel any file generation in the Downloads dropdown by clicking on the cross icon.

As well as downloading the segment report, you can also download the segment report + filter.

If you want to delete any downloaded files simply click on the trash icon to remove them from the downloads dropdown.

Graphs

Top segments graph

The top segment graph displays the top five segments that have the most URLs in a report. It can be used to quickly highlight problem areas of a website in any report.

The data points are clickable so you can simply click on any segment in the graph and it will take you to the segment in the report (which will persist when you move to another report in the new core app).

Segment report chart

The segment report chart compares the currently selected segment URLs against all other URLs in this report. It allows customers to see how big the URLs are within the context of the whole report.

If you have only created a handful of segments and these segments are quite small in the context of the rest of the pages in a report, then it indicates that more segments need to be created.

Trend chart

The trend chart maps out the data trends over the last 30 crawls for any segment report. It can help you track improvements or issues for particular areas of the website.

For example the number of pages over the medium fetch time being crawled spiked in the last crawl in the /2020/ area of a large publisher website.

Hints and Tips

Regex validation

If you choose Matches (Regex) and Doesn’t Match (Regex) rules, please enter Lucene valid regular expressions. Use Rubular to help write valid regular expressions. Forward slashes do not need to be escaped.

Managing and copying segments

1. You can delete a segment, a tooltip will always ask if you want to delete to ensure you don’t accidentally delete a segment
2. You can view rules for each segment created
3. You can see when a segment was generated
4. You can bulk copy a group of segments to another project in your account.

Author

Helen Cherry