The Value of AB Testing UX Designs

What is AB Testing?

AB testing, or sometimes also referred to as split testing, is similar to an experiment, where two versions of a design such as a website homepage or an app screen are compared side-by-side to determine which one is more successful. To ensure fairness, both versions are made available at the same time and exposed to the same target audience, with inbound traffic split between each. A portion of the audience will be directed to the first version and the rest to the second one.

The test measures the design which is more successful based on a certain set of criteria such as conversion rate or bounce rate for a webpage. The criteria set is based on what the test is trying to achieve, for example, if a company wants to use AB testing to see if a more prominent call-to-action (CTA) button can lead to more users clicking on it, they can measure the click rate between different designs.

The Importance of AB Testing

AB testing is a great UX design strategy as it provides people with valuable data on whether certain features or designs have an effect on the users, in order to create a better user experience journey. It forces designers to look past their own biases, preferences, and ego, and instead focus solely on the results and behaviour of the users.

The data obtained from each group between the versions allows companies to make informed decisions backed by evidence. It allows companies to test out a new design approach or change, before fully committing to it. AB testing is also versatile in the sense that it can be applied to many different aspects such as marketing, user experience, and design.

Benefits of AB Testing

1. Better user experience

Besides user research and testing, AB testing can further help to figure out the most effective solution by using user’s response to different designs. The data collected through AB testing helps in making better decisions that would improve the user’s experiences.

2. Better return on investment (ROI)

An improvement in a customer’s user experience journey also correlates to better ROI as key metrics such as bounce rate and conversion rate improves as they are able to navigate easier or understand the value of the company better. Constant re-iteration and updates through AB testing ensures a design’s effectiveness.

3. Quick and easy

AB testing is fairly straightforward as determining the results is a matter of comparing both page’s metrics against the set goal e.g. bounce rate. The tests themselves do not take long to set up as they can be implemented through an app or webpage that is already live. Furthermore, even a small sample size from AB tests is able to provide effective, actionable results. There are also many easy-to-use, accessible AB testing platforms available on the market such as ABTasty.

Here are some examples of situations where AB testing is used to improve the user experience strategy.

AB Testing Methodology

Here is a guideline you can follow on how to use AB testing on your designs, in order to create a good user experience for your users.

1. Identify your goal

You should have a clear objective for your AB tests, such as increasing the homepage conversion rate or increasing time spent on a certain page. Having a goal helps you to establish which metrics to measure and how to create your different designs to test against one another.

2. Formulate a hypothesis

A hypothesis acts as the basis for the AB test as it allows testers to focus on certain factors to test and compare the different designs. To generate a valid hypothesis, one should consider why and how a certain design variation would be more effective. An example hypothesis could be: Changing the call-to-action (CTA) button text from “Download” to “Download Career Guide” would increase downloads as users are more clear on what they would receive.

3. Set testing criterias

Identify the key metric to track in line with your set hypothesis and goals. For example, when looking to test the effectiveness of a different copy, you will probably be tracking bounce rate or time on page. However, when tracking effectiveness of a CTA button text, tracking button clicks may be more relevant.

4. Design and carry out the test

Create two variations of your design with your AB testing tool. Be sure to have a control design and refrain from changing too many factors when starting out as it can be difficult to pinpoint what affected the metrics.

5. Analyze and implement

Lastly, carry out and run the test over a set period of time. The timeframe required is dependent on your site’s traffic. Once completed, collect the data, analyse your results, and implement the necessary changes. Do note that having a larger sample size is more representative of the users.

AB Testing Tips

Now that you have learned the basics of AB testing, here are some additional key tips to help bring your AB test even further!

Multiple variant testing

When running a standard AB test between two variants, your observations and results are usually limited to whether your new variant does better or worse than the original. However, with multiple variant testing, you can test more than two designs at any time, simultaneously testing multiple hypotheses.

Effectiveness with higher traffic

A/B testing is faster and more effective if you have higher traffic — ideally more than 1000 unique visitors a month. The amount of traffic on the pages you are AB testing for is important to obtain better results and maximise your returns.

For sites with lower traffic, a tip could be to test things with a higher impact instead, in order to maximise the results and obtain actionable insights. When testing an aspect with a higher impact, the difference in results can reduce the time taken to get statistically significant results. Additionally, you should also avoid multiple variant testing with limited resources.

Calculating statistical significance

This brings us to our next point, calculating statistical significance. Statistical significance is important as it proves the level of certainty of the test results, making sure that they are not due to sampling errors or random chance.

Popular software

Last but not least, some popular software that you can consider when AB testing includes:

• Google Optimize
• Optimizely
• AB Tasty (CuriousCore’s partner)

Conclusion

I hope that this article has provided you with some insight into why AB testing UX designs and why it is so useful in creating good UX designs as well as a brief idea on how to establish an AB test. If you want to know more about user experience and its industry, do check out our other resources available on our website, such as our articles, weekly webinars, and podcasts.

CuriousCore offers both a 2-day UX course and an intensive 4-month career accelerator course for those keen on transitioning into the industry. Click the buttons below to find out more.