Artios is a digital marketing agency that uses mathematical insights to drive sales for their clients. They use DeepCrawl to gather data that they then feed into their analytics platform.
Thanks to DeepCrawl’s API capabilities and product features, the Artios platform is able to predict competitor SEO advantages to a 95% accuracy rate, and then quantify what each ranking factor is worth. This helps them prioritise these recommendations.
Andreas and his team wanted to find a way to probe their competitor’s SEO behaviour, which they could then feed into their machine learning platform. DeepCrawl is data driven, has API support and is extremely thorough, making it the perfect choice. The Artios team also liked that DeepCrawl were adding new features to its platform, which meant more potential factors that could explain and predict rankings in Google.
Artios specifically needed an industrial website crawler that could:
- Gather data on a daily basis for statistical resampling;
- Handle large websites for ecommerce clients;
- Crawl on a stealth basis;
- Provide numbers (and deltas) on SEO features like number of words;
- Be reliable so as to avoid disruption thereby creating gaps in their data modelling.
DeepCrawl had the potential to provide Artios with a way to gather data that was representative, consistent and accurate. This would empower them to make predictions in SEO that would go far beyond industry best practice.
Thanks to DeepCrawl’s API, Artios now has access to DeepCrawl features such as word counts and 301 redirects, allowing them to collect competitor data on a regular basis. All they need to do is set the client and competitor domains in the DeepCrawl interface.
Using the API, Artios can then set up scheduling for their domain
Artios use the DeepCrawl API every day, at random times of the day and to research multiple competitors and compare them to their clients. It’s very flexible, and they wanted to avoid set patterns while making the samples unbiased.
The Artios platform then stores DeepCrawl’s data in a cloud database. Their data mining algorithms perform Exploratory Data Analysis (EDA) to spot and quantify interrelationships between SEO variables as well as rankings. This also allows the transformation of data into normalised distributions, which comes in handy when they need to perform linear regresssions when making predictions later on.
Using DeepCrawl data via the API on competitors as well as clients, means Artios can now compare how each site in a client’s market sector compares across each SEO feature:
The graph above shows the leading competitors are using significantly more words (there is a less than 0.00001% chance of the client’s pages having the same word count as hitched.co.uk) for the web pages. It also suggests how many words the client needs to target on their web pages: in this case 1,500 words.
Performing linear regression on the SEO feature for the market comparison also enables Artios to see how likely the SEO feature is to shift rankings:
The graph above shows that the coefficient of determination is 45.3%, which means that there is a high probability that a client dedicating resources to increase word count to 1,500 words or more will increase rankings.
If you are not yet using the DeepCrawl API, you will find the word counts peer page in the Minimum Content Size report, as shown in the screenshot below.
Artios algorithms repeat this process for each and every feature offered by DeepCrawl, such as:
- Average number of words per URL;
- Volume and proportion of 301 redirects to overall site pages;
- Volume of incomplete Open Graph tags;
- Average Deep Rank;
- and many more.
This then allows Artios to construct a model for predicting and quantifying technical SEO changes that will increase rankings. Often, they see jumps in rankings as a result:
The graph above shows the ranking impact of responding to analysis based on DeepCrawl data gathered on a client and their competitors. In the above case, Andreas and his team increased the average word count to 1,000 words per page on existing pages and instilled this as part of the content marketing policy, to increase our client’s rankings. This insight was made possible by Deep Crawl’s data and its availability via the API.
The machine learning kicks in when Artios compare their model’s predictions against the changes made onsite and the resulting ranking improvements. The Artios platform’s predictive modelling capabilities improve by adjusting the weights in the model, for example adjusting the number of ranking position increases due to the change in the number of words made by the client.
This machine learning aspect means that instead of helping their clients with a one-off jump in ranking, Artios can keep ensuring their clients stay ahead of the field.