Using data to optimise search engine optimisation

In the context of SEO, a Data Scientist can help to better understand search and traffic data, enable ‘real-time’ optimisation research and predict the results of SEO strategies. To do this, they have a number of analytical tools at their disposal.

Web page optimisation

Logs are the computer traces left behind when a page is visited. The work of the data scientist will enable prioritisation of pages to be closed or redirected, automated analysis of log files, detection of trend breaks and error peaks. Data visualisation approaches can be used to provide results in the form of an alert system or dashboard.

The preliminary analysis will enable the data to be classified, distinguishing between visits by crawlers, search engine indexing and user visits. Google’s crawlers, as well as its competitors, have user-agents with predefined IP addresses, which enable them to be identified in the mass of visits.

The logs can then be studied using various approaches:

• highlight pages returning errors (404 error for a deleted page, etc.),
• detect pages with abnormally long loading times,
• or tracking pages that have had few or no visits in relation to the crawl time they consume.

It is better to save this crawl time for other pages with higher added value.

Based on the site’s page inventories and structure, we can better analyse the reasons for the performance of certain pages. Using a few queries, the data scientist can isolate pages that are orphaned or poorly linked.

It is now possible to anticipate the possible results of a search engine optimisation policy. Data can be used to monitor the emergence of a competitor, new trends or even new products.

Web scraping methods can be used to automate the journey through a website. In the same way as search engine crawlers scour the sites to be referenced, it is possible to frequently retrieve the SERP for a list of predefined keywords.

This gives the data scientist multiple avenues of analysis: he can analyse positions at a certain point in time, but also temporal variations by historising ranking data. With a bit of inventiveness, he can propose a control chart and put in place management rules with the SEO experts so as to be alerted as quickly as possible in the event of an incident on the site. By counting, they can see the emergence of a competitor.

In addition, the Data Scientist will be able to automate and industrialise certain tasks in the search for new keywords and to record some of these keywords. They can be used to highlight new services or products, differentiating factors through the creation of new pages on the site, or simply to create a pool of future opportunities. Just think of a keyword like “external phone battery” and the websites that sniffed it out first.

Audience tracking tools, such as Google Analytics or Matomo, give data scientists access to all the data they need to create dashboards that facilitate decision-making and responsiveness.

They will be able to create customised graphs based on tracking user behaviour and facilitating adhoc developments in commercial environments in particular. In this way, users can be tracked page by page, or even within each page. For an e-commerce site, the Data Scientist will create a conversion tunnel and provide an understanding of what happens at each stage of the transaction by cross-referencing visitor data with logs, for example.