Our rendering engine is built to be as flexible as you need it to be. To help enable you to do whatever custom analysis you'd like to do on your website, it allows for custom Javascript files and snippets to be executed on every page we render. 

Performance Statistics

Chrome, our rendering engine, gives us access to a whole host of page rendering performance statistics via Javascript.

Some of these are:
Performance timings
  • navigationStart
  • unloadEventStart
  • unloadEventEnd
  • fetchStart
  • domainLookupStart
  • domainLookupEnd
  • connectStart
  • connectEnd
  • requestStart
  • responseStart
  • responseEnd
  • domLoading
  • domInteractive
  • domContentLoadedEventStart
  • domContentLoadedEventEnd
  • domComplete
  • loadEventStart
  • loadEventEnd
Read more about the performance timing numbers Paint Timings
  • first-paint
  • first-contentful-paint
Read more about the paint timing numbers Resource Statistics
  • resource-count: Number of resources used in the page
  • resource-transferSize: Total amount of bytes transferred for page resources
  • resource-decodedBodySize: Total amount of bytes that resources used in memory (after being decoded/decompressed)

Capturing these statistics

While we plan to include these metrics in DeepCrawl by default soon, a quick method to get these into your crawls now is using our custom functionality. Once you've decided which performance metrics you'd like to see in your rendered crawls, we need to capture these using a two-step process.

Grab the metrics using a custom Javascript injection
Add the following code snippet to the "Custom Script Injection" setting of your project: 
var perfTimings=performance.timing.toJSON()
Object.keys(perfTimings).forEach((i)=>{
  if(perfTimings[i]>0){
      var newSpan = document.createElement("span");
      newSpan.id = "customextraction-perftimings-"+i
      newSpan.innerText = ((perfTimings[i]-perfTimings.navigationStart)/1000).toFixed(2)
      document.body.appendChild(newSpan)
  }
})
performance.getEntriesByType('paint').forEach(entry => {
          var newSpan = document.createElement("span");
      newSpan.id = "customextraction-perftimings-"+entry.name
      newSpan.innerText = (entry.startTime/1000).toFixed(2)
      document.body.appendChild(newSpan)
})
var deepCrawlResourceMetrics = {count:0,totalTransferBytes:0,totalDecodedBytes:0}
performance.getEntriesByType('resource').forEach(entry => {
    deepCrawlResourceMetrics.count++
    deepCrawlResourceMetrics.totalTransferBytes+=entry.transferSize
    deepCrawlResourceMetrics.totalDecodedBytes+=entry.decodedBodySize
})
Object.keys(deepCrawlResourceMetrics).forEach((i)=>{
    var newSpan = document.createElement("span");
    newSpan.id = "customextraction-resources-"+i
    newSpan.innerText = deepCrawlResourceMetrics[i]
    document.body.appendChild(newSpan)
})
Capturing these statistics with a custom extraction
The above step will dump the performance statistics into the DOM of your pages as we crawl them. The next step is to extract these into custom extractions.

Depending on the metrics that you'd like to save, use the following custom extractions:
  • navigationStart
  • unloadEventStart
  • unloadEventEnd
  • fetchStart
  • domainLookupStart
  • domainLookupEnd
  • connectStart
  • connectEnd
  • requestStart
  • responseStart
  • responseEnd
  • domLoading
  • domInteractive
  • domContentLoadedEventStart
  • domContentLoadedEventEnd
  • domComplete
  • loadEventStart
  • loadEventEnd
Read more about the performance timing numbers Paint Timings
  • first-paint
  • first-contentful-paint
Read more about the paint timing numbers Resource Statistics
  • resource-count
  • resource-totalTransferBytes
  • resource-totalDecodedBytes

Using this data

Once your crawl has run, these metrics should be available in both the "Custom Extraction" reports, and the "All Pages" export. 

The performance metrics are in seconds from the beginning of the render.