A/B testing is a process based on a culture of experimentation where the end game is to determine the best marketing and promotional tactics, especially online. A/B testing is used to fine tune ad messaging, copy and images on websites, online forms or emailings. The benefits of a solid A/B testing programme are many– by selecting the images, elements and messages that reflect user preference, the results of A/B testing can positively impact your business and your bottom line. In this post, we’ll explore some basic A/B testing terminology and some of the associated benefits and practices.
What is A/B testing?
An A/B test is a way to optimise web pages, messages or ads. In the case of a web page, let’s imagine that we have a version A and a version B. Both pages contain exactly the same offer or promotion and their difference lies in their design. Suppose we have a landing page that offers a 10% discount on our products if a user signs up for our newsletter– version A has yellow sign-up button and version B has a red button. Both versions are served to an equal minimum number of users so that the results are statistically significant, and subsequently, the version of the page with the highest newsletter conversion rate would be the “winner”.
A/B testing can be performed with individual elements (as in the example above where both pages are identical, only the button colour changes). With this type of test, we generally change one element at a time, test on that alternation, and then continue with other elements on the page. For example, if the yellow button on the version A page has higher conversion, we keep that style button and then move on to testing two different titles in our next round of testing.
We can also change all elements on a page; imagine that version A has a yellow button but also a video and almost no text, whilst version B with the red button has lots of text but no visual elements.
The idea behind A/B testing in both cases is to let our decisions be guided by user preferences and measure user behaviours with an objective in mind (conversion rate, clicks, etc.) rather than depending on our own suppositions or opinions based on personal preferences instead of data.
What’s the difference between A/B testing and multivariate testing?
A multivariate test is similar to an A/B test, just a bit more complicated. Multivariate testing uses the same mechanisms as A/B testing, but involves testing multiple variables at the same time, and the relationship between these variables (i.e. that yellow button performs better than the red one, but it performs even better if it is accompanied by extensive an copy on-page). A multivariate test measures the effect of each design combination in reference to a specific objective.
For our purposes here we’ll focus on A/B testing, but a good online marketing programme can benefit from a nuanced multivariate testing plan to improve web or business results. At All Around, we’re experts in multivariate and A/B testing and we can get your business up and running quickly with a testing programme designed to improve business results.
A/B Testing is easy and it works.
Perhaps the most compelling aspect of A/B testing is that it is extremely easy to implement and provides actionable insights almost immediately. For example, comparing two Google Adwords text ads can quickly indicate that your users have a clear preference for promotional texts that include the final price rather than a percent discount. A higher CTR or conversion rate in the ad with a final price quickly tells us that we can write more ads that include final product prices and our results will likely improve.
Experimentation without big risks.
Maybe the CEO of your company is determined to redo the entire website: he or she believes that more aggressive messaging and a black background on the site will improve business results. The CEO could be right, but he or she is making decisions based on personal opinion, not hard data. Overhauling and redesigning the whole website also will incur costs, manpower, and potential economic risk. A bit of of A/B testing with a landing page sporting a design similar to what the CEO wants to implement and another landing page with the more traditional design can give you a clue of the proposed changes are really a good idea or not.
You can test anything!
A/B testing need not be limited to web pages or online forms. You can A/B test offline as well with the use of unique telephone numbers or URLs. For example, if you create a yellow flyer and a red flyer containing the same information but with a different phone number on each, you can track which colour flyer has resulted in more impact (calls in this case). Just like online, it’s important to have an equal and representative sample of people receiving each version– if you distribute 500 yellow flyers an only 10 red ones, the test will be stacked against red and will not be representative.
A/B Testing can be used for more than just conversion.
A/B testing is very useful if you are optimising for conversions or traffic. But it can also be used to optimise for other types of user behaviours. For example, your web is complicated, and major changes involve some serious operational challenges– an A/B test can help you to determine if a certain style page will lower the bounce rate or result in users spending more time on that page. These pieces of information can be useful when it comes to improving user experience on your site, not only conversion rates.
If you are interested in implementing A/B and multivariate testing to improve business results, contact us.
More posts about: Increase brand awareness
Client Success Manager
Alexis is our Client Success Manager at All Around. She is a Digital Marketing Specialist and Manager with 15+ years of experience in international markets. She is a dedicated client partner, ready to take businesses to the next level, with vast expertise in digital strategy, operations, project management, reporting, budgeting and negotiation.
