Website Optimisation – A/B and MVT Testing
It’s not always easy to identify what effect changes to your website will produce. The two techniques described below can help with this process and will hopefully lead to your website being more effective (and popular).
How should I test changes to my website?
A/B and MVT testing are similar in execution. You create two versions of a page that you hope to optimise (perhaps a checkout page, or newsletter signup), modify one of the pages in a way you believe may improve it’s performance and provide both pages on your site to different users.
- Create a copy of one of your existing pages that you aim to optimise (checkout page, newsletter signup etc)
- Modify the second copy in one or more ways that you hope will improve performance
- When a user requests that page provide one or the other of the pages.
Using a tool like Google Analytics, you can then monitor the performance of the two pages in regard to conversions, signups etc and hopefully identify which page is performing better.
You need to monitor the testing for a period of time, as the larger the sample set the more accurate the results.
The key difference between the two methods is that A/B testing uses a page that has a single specific change i.e. position of a PayPal button, whereas in MVT testing there will be a number of changes between the two pages.
In most other respects the two methods work similarly.
What do I need to know about A/B testing?
Advantages of A/B Testing
- Ease of setup – a single change in position of a button or the size of an image can be done relatively quickly.
- You know exactly what is working and what isn’t – with a single change you know that if your page is performing better it’s likely because of that one change.
Disadvatages of A/B Testing
- Multiple changes will be slow to test – as you test each change indivdually, if there are a number of changes to a page it will take a long time to complete each test.
What do I need to know about MVT?
Advantages of MVT
- Multiple changes can be tested at once – as you test a number of changes at the same time, this will be far quicker to test than a number of A/B Tests.
Disadvatages of MVT
- Reduced clarity of results – multiple changes at once means that you can never be sure exactly what changes are working and which changes are not. Some changes may even be benefitting the page, while others may be hindering it at the same time obscuring your findings.
Choose your testing method based on your time limits
Which method you use, depends on how much time you wish to expend optimising the page and what changes you plan to make.
A/B testing will allow you to determine exactly what elements benefit the page, but to test a large number of elements will be very time consuming. MVT testing will be far quicker, but may potentially obfuscate some of the benefits/deficits of your changes.
In general, A/B testing is likely to work for most sites short of a radical page rebuild. It is possibe of course, to make significant changes, do a MVT test, and then further optimise this with a few focussed A/B tests to further optimise specific areas.