How to set up a Split URL test
As we mentioned before, Split URL tests have a larger scope than A/B tests. They are more flexible because the parameters you can change and test are more diverse. Keeping that in mind, you can look at an existing web page and analyze it beyond just its UI elements. But before we dive into the benefits of this approach, let’s start with how to set up a test:
Identify the web page you want to optimize
Make a list of the problems you see with it. Ask questions like: What are my goals for this page? Why do I feel those goals aren’t being met right now? What are the aspects of this page that I feel could be better?
Put together all the information you have about the web page. Pull together statistics of user behaviour, check analytics, and ask for opinions. Correlate with user feedback through various support channels.
User feedback plays a huge role in designing a great website or product. Implementing the opinions of existing users increases their tendency to identify with your product, and therefore their inclination to use it.
Formulate a hypothesis
Make an assumption based on your insights: if you change X, Y will happen as a result. This hypothesis becomes the framework of your test, and how you determine whether or not it was a success.
Structure your hypothesis in the form of if-then statements. For example: if I change the layout of the web page, then it will be easier to read and therefore the bounce rate will reduce.
Build an alternative
Design a new web page using all the information at your disposal, finding solutions to better reach your goals. Since a split URL tests can accommodate much bigger changes than A/B tests, you can consider making changes like altering the workflow of a group of pages. For example: can the checkout process be easier? Should the cart open on the side of the page, rather than take the user to an entirely new page? And so on.
Define success metrics
Identify the KPIs you want to track for the test. Typical KPIs for split URL tests include conversion rate, downloads, and bounce rate, among others. Match these to the relevant parameters for the test.
It is important to let the test run for an adequate amount of time. There are many one-off occurrences that can skew the results of a test, and time is the only way to counteract any spikes. This is known as allowing the test to reach statistical relevance.