Measure Core Web Vitals and Web Performance Metrics in DeepCrawl

Adam Gent
Adam Gent

On 29th September 2020 • 9 min read

With Google’s recent announcement that Web Vitals metrics will become a ranking factor, SEOs need to be looking to expand their performance testing. DeepCrawl now allows our customers to capture web performance and Core Web Vital metrics automatically into their crawls.

Important: Because Web Vitals are measurements of how a page renders, a crawl must have JavaScript rendering turned on to have these metrics and reports.
 

What are Core Web Vitals?

The Core Web Vitals are a set of web performance and user experience metrics that are based on real-world usage data from Google Chrome. The Core Web Vitals metrics are:

Earlier this year, the Google Webmaster team announced that Core Web Vitals will be part of the Page Experience ranking update in 2021 (no date confirmed yet).
 

How to incorporate DeepCrawl into your site speed toolkit

DeepCrawl automates capturing Web Vitals and web performance data at scale in a controlled environment. Because we are testing a page’s vitals in our own systems, this type of performance data is called lab data. Both Lighthouse and other performance testing tools capture this type of data. 

The performance and Web Vital metrics in DeepCrawl are just another tool in your site speed toolbox. We recommend combining any lab data (DeepCrawl, Lighthouse, etc.) with other real-user monitoring (RUM) data to track the page experience of your users.

Our team recommends reading more into lab vs RUM data for site speed tools, the Chrome UX report, and creating your own CrUX dashboard.
 

What new metrics are available?

DeepCrawl have added the following performance and Web Vital metrics:

If you are unsure about tracking page performance or Web Vitals, then read our guide on how to measure site speed and performance.
 

What new reports are available?

DeepCrawl has added reports for each new performance and Web Vitals metric, these new reports can be found under the Performance category.

web-vitals-in-navigation

Each new performance or Web Vital metric is broken down into Good, Average, or Slow (Poor for CLS).

Largest Contentful Paint layout

The breakdown of Good, Average, Slow (Poor for the CLS metric) pages for each metric is based on the Lighthouse performance audit scores.

A summary of these performance scores can be seen below.

 

Metric Good Average Slow
Time to First Byte 0 – 200 ms 200 – 600 ms Over 600 ms
First Contentful Paint 0 – 2 secs 2 – 4 secs Over 4 secs
Largest Contentful Paint 0 – 2.5 secs 2.5 – 4 secs Over 4 secs
Cumulative Layout Shift 0 – 0.1 0.1 – 0.25 Over 0.25
Time to Interactive 0 – 3.8 secs 3.9 – 7.9 secs Over 7.3 secs
Dom Content Load 0 – 2 secs 2 – 4 secs Over 4 secs

The performance score criteria for each report can also be found in the information icon next to the report title.

Information about reports
 

How can I get started?

Any customer can access these metrics, all that is required is that the JavaScript rendering feature is enabled and the metrics will be captured in your next crawl.

JS-rendering-feature-enabled

That’s it. DeepCrawl will then capture the performance data and populate these reports.
 

Frequently Asked Questions

How does DeepCrawl measure Core Web Vitals?

DeepCrawl captures the Core Web Vital and web performance metrics using our Page Rendering Service (PRS). The page rendering service uses the most up-to-date version of Google Chrome for rendering JavaScript.

As DeepCrawl crawls and renders the HTML of each a page it captures the Core Web Vital and web performance scores in a lab environment. These performance scores are then treated like any other metric in our system.

Why doesn’t DeepCrawl match up with Google site speed tools?

The performance data captured in DeepCrawl is in a controlled environment, which uses a different technology stack and runtime settings to Google’s and other third-party site speed tools.

Does DeepCrawl use Google Lighthouse or the PageSpeed Insights API?

DeepCrawl does not rely on third party APIs to capture the Core Web Vitals data. All the metrics are captured using the most up-to-date version of Google Chrome.

Why isn’t First Input Delay (FID) captured in DeepCrawl?

The Web Vital metrics in DeepCrawl are captured in a lab environment and not real user monitoring data from the Chrome UX report. Google has stated that FID requires “a real user to interact with the page, and thus cannot be measured in the lab”.

Our team recommends creating your own CrUX dashboard to measure FID across your website.

What are the runtime settings? 

The runtime settings for DeepCrawl when capturing data:

Is this data available in Data Studio?

All of these reports are now available in our Data Studio connector. To add these reports to an existing dashboard, you must edit the data source and “Reconnect” it to allow it to find these new data points.

Will the new metrics and reports be available in the API?

Yes. Please read our API documentation to understand how to start pulling the data into your own workflows.

How can I improve my performance or Web Vital scores?

Our team recommends reading our web performance best practice guide to identify ways to improve the site speed of your pages.

Will I still need custom extraction reports for performance data?

Our team recommends reviewing all the new performance reports and if they meet your needs, remove any unnecessary custom extractions.
 

Summary

The addition of performance and Web Vitals metrics and reports to DeepCrawl will allow our customers to automate capturing and monitoring lab data at scale, ready for the Page Experience update in 2021.

Any questions please feel free to get in touch with our team.

Author

Adam Gent
Adam Gent

Search Engine Optimisation (SEO) professional with over 8 years’ experience in the search marketing industry. I have worked with a range of client campaigns over the years, from small and medium-sized enterprises to FTSE 100 global high-street brands.

Get the knowledge and inspiration you need to build a profitable business - straight to your inbox.