A complete redesign of a website or migration to a new platform can offer opportunities for improved functionality and aesthetics, but it also brings significant risks for conversions. This is mainly because customers are accustomed to the existing structure and functionalities of the website. Here are some of the biggest dangers and why companies often choose a redesign, along with tips to minimize conversion loss through split testing.
Image explanation: Conversion loss after a redesign of 150 million by Marks & Spencer. The site was quickly adjusted after complaints about confusing navigation, leading to an 8.1% revenue drop in the first quarter after launch.
Why Companies Often Opt for a Redesign
Many companies combine their platform migration with a complete redesign for various reasons:
- Outdated Technology: Companies often want to implement more modern features and technologies to enable better performance and scalability, such as expanding with new features.
- Improved Customer Experience (UX): A redesign offers the opportunity to implement UX updates, such as faster load times and improved navigation.
- Brand Repositioning: Some companies use a redesign to reposition their brand or renew their visual identity.
- Competing in a Changing Market: Technological innovations such as Generative AI and Spatial Commerce drive companies to redesign to remain relevant.
While these reasons are legitimate on their own, they can become problematic when combined with a radical change in user experience and navigation. Customers accustomed to a certain way of working can become overwhelmed, leading to lower conversions or even customer loss.
The Risks of a Complete Redesign:
- Undermining Customer Trust: A sudden change can confuse customers, especially if they have trouble finding their way in the new design. This can lead to higher bounce rates and lower conversion rates.
- Technical Issues: During the transition to a new platform, bugs, errors, and slow load times can occur, deterring customers.
- SEO Impact: Major changes in URL structures and content can negatively affect existing SEO rankings, directly impacting organic search results and thus traffic.
- Inconsistencies in Functionality: If certain elements, such as payment methods or product filters, do not work smoothly in the new design, it can cause frustration among customers.
Split Testing as a Solution to Minimize Risks
It's important not to switch to a completely new design without first testing the impact. You can do this through usability tests or by using split testing (A/B tests). With a split test, you can gradually present a new platform or design to customers and monitor how this affects customer behavior and conversion.
An example of how companies like Hunkemöller have followed this approach is through A/B tests. The following actions are required for this:
- A split script and cookies: To optimize the customer experience, a split script is used that splits visitors across the old and new site. This is often done via cookies that maintain the customer's assignment to a specific version (A or B), ensuring consistency in their experience.
- Traffic Distribution: A small percentage (e.g., 10%) of traffic is directed to the new site to quickly detect and address potential issues without affecting the overall performance of the site. After resolving the major issues, you can scale this up to 50/50 to compare behavior patterns and conversions.
- Tracking and Analysis: All interactions on both versions are tracked with analytical tools like Google Analytics 4 (GA4), providing insights into key performance indicators such as conversion rate, average order value, and customer interactions.
The 3 Biggest Risks in Split Testing
The challenges that this method brings are how you will distribute the traffic? Will you use an existing experimentation tool like Kameleoon, Convert, VWO, or ABSmartly? Will you split the traffic via a script in Google Tag Manager? Or will you set it up yourself on the server side, such as via Salesforce Commerce Cloud or Optimizely?
- Delay or Flickering When Redirecting Traffic: Split testing via an experimentation tool or your Tag Management system has a downside. These scripts are loaded when the page loads. This means that the original page must load to redirect customers to the new page. Sometimes this means that a customer sees a flicker effect. The flicker effect occurs when a page is loaded and the customer sees the changes of a test happening on the page. You can solve this in 2 ways; 1. Ask the customer if they are open to using the new version of the platform with a link to the new environment
2. Use the script server-side. This means that users are directed to the new version before the page loads.
-
Consent Mode and Cookies: Before seeing an experiment, one must agree to the cookies. However, the cookie notification is a notification you must give before you can use experimentation tools and experiment with customer approval. If you test server-side or within your own CMS, you can redirect customers to the new version before the original page loads. It is important to still ask for the cookie notification on the new version.
Because a split test uses a cookie so that customers also land on the correct version via a search action via Bing or Google, it is important to set this cookie correctly. If your customers do not always convert in the first session, you cannot use a session cookie but a cookie of at least 30 days.
- Analysis of Customer Audiences: Returning customers have already seen the original version of the platform. This makes them biased or, in English marketing terms, 'Biased'. If they are loyal customers, they know their way around the current platform easily, so old expected patterns can influence behavior on the new platform. If loyal customers have a great value within your organization, it is important to consider this in the design and analysis.
A separate account for experiments offers the opportunity to provide customers with a consistent experience across all their devices. Without this approach, you risk a customer seeing a different experience or version on mobile than on a laptop.
A Test Plan with Acceptance Criteria and Success Criteria
A solid test plan includes acceptance criteria. These are conditions that must be met for a split test to be successful:
- No Negative Impact on SEO: All pages must remain the same in terms of URL structures. The new pages must be protected from indexing by search engines during the test to avoid affecting current positions. It is therefore important to also block bots in the experiment.
- Key Health and Behavioral Statistics: Besides the conversion rate, other behavioral statistics such as the number of items added to the cart and the average order value must remain stable or improve.
- Measurability: If you want to measure a difference of 10%, this means you want to see if there is a difference in conversion rates that is at least 10% higher or lower than the baseline. As much as you want to see an increase in revenue and transactions, when replatforming, the goal is often to at least maintain the same number of conversions to build on. The smaller the difference in conversion rate you try to detect (e.g., 5% instead of 10%), the larger the sample size must be. This is because reliably detecting small differences requires more data to reduce the margin of error.
- Example 1: If your conversion rate is 1% and you have 100,000 visitors per month, you can measure a difference of +/- 5%.
- Example 2: If your conversion rate is 2% and you have 50,000 visitors per month, you can measure a difference of +/- 10%.
Therefore, always conduct a pre-test analysis.
- Conversion Rate (CVR): The new platform must achieve at least the same conversion rate or not fall further than -X% compared to the old site to be considered successful.
If these criteria cannot be met, it means you depend on qualitative research such as usability testing, user interviews or panel discussions.
Conclusion
A complete redesign combined with a migration to a new platform can be a risky undertaking, especially regarding conversion loss. Companies choosing this strategy must ensure they test step by step with a structured approach, such as split testing, to monitor the impact on customer experience and revenue. A successful migration and redesign depend on constantly measuring performance and iteratively improving the design based on customer behavior.
