Every two weeks, senior SEO consultant Koen Leemans writes about an interesting SEO A/B test case. The tests are conducted in collaboration with Semrush and their testing tool SplitSignal. Prior to the test, we asked the Twitter followers of SplitSignal for an estimation of the result. According to them, what is the effect of adding category names to product page titles within an e-commerce website?
The majority of followers were wrong! Instead of an increase in clicks, we saw a decrease as a result of the change. Read on to find out why.
The case
Page titles are often the most prominent element a user sees in search results. The competition with other search results is significant. Therefore, it is essential to write more relevant page titles than your competitors. Regular testing helps you find the most optimal page title. It is also important to find the right balance between using relevant keywords and page titles that attract users' attention to improve CTR.
Product pages on an e-commerce website are often the pages that generate the most revenue. Testing within these pages with the aim of finding the best possible answer for search engines and users can result in better organic performance and more revenue. For a large e-commerce party, we therefore added the name of the product category to the titles of product pages. The aim was to determine whether this leads to more relevant and effective search results.
The hypothesis
Our hypothesis is that adding the category name makes the page title more relevant because it gives users and search engines more insight into the product type. Additionally, adding the category name provides more relevant keywords related to the product. The expectation is that this increases visibility for relevant search queries.
With this test, we validate whether adding the name of the product category to the product page title has a positive effect on organic traffic. The example below shows how the names of product categories have been added to the page titles.
The test
With SplitSignal, we changed the page titles of the tested product pages. In total, we selected approximately 5,000 pages for this, divided into the control and variant group. The test was live for 21 days during which Googlebot visited 91% of the tested pages.
A script monitored during the test period whether Google showed the desired changes of the page titles to users. We established that the changes were indeed visible in the search results.
Results
Adding the name of the product category to the titles of product pages resulted in an estimated decrease in clicks of -8.2%.
Within a week, we could determine that the measured effect is significant. When the blue-shaded area performs above or below the x=0 axis, the test is statistically significant with a confidence level of 95%. This means we can confidently say that adding the product category to product page titles has a negative effect on organic traffic to product pages within the website.
Note: We do not compare actual pages from the control group with pages from the variant group, but make a prediction based on historical data. The model predicts the counterfactual response that would have occurred if no change had taken place. We compare this with the actual data. We use a set of pages from the control group to give the model context for trends and external influences. If something else changes during our test (e.g., seasonal influences), the model detects this and takes it into account. By filtering out these external factors, we gain insight into the true impact of an SEO change.
Why?
This test perfectly illustrates why it is important to validate changes before implementing them on your website. In this way, testing acts as a safety net against optimizations that negatively impact your organic performance. But why does this specific optimization lead to a decrease in organic traffic? We, of course, investigated this for you.
Google truncates long page titles
Firstly, many page titles became too long, causing Google to truncate them in the search results. Because the tested website is from a well-known brand, this may have contributed to the negative result.
Product categories are redundant for long-tail searches
We also concluded that more descriptive information about a specific product type is likely redundant for Google and users. Generally, Google shows the same results for searches like “bose QuietComfort 45 black” and “bose QuietComfort 45 black headphones”. This shows that Google already understands which product it is and for which searches the related product page is a relevant result, even without mentioning the product category. Additionally, users who enter such long-tail searches already know they are looking for, in this example, headphones. Adding the product category to product page titles therefore adds no extra value for the user.
To further validate the findings, we conducted this test for multiple websites. All these tests had a negative result. For an online music store, this resulted in a decrease in organic traffic of -7.2%. A third e-commerce website showed a loss of -5.4%. This does not mean that adding the product category to product page titles will necessarily have a negative effect. However, it does show that such a change should always be tested before implementing it on your product pages.
Remember that a change that works for one website does not have to have the same effect on another website. Only by testing will you always know what works for your website!
```
