A/B testing, or split testing, has been part of the marketing landscape for years, if not decades. With the advent of technical marketing, growth hacking and engineering marketing, A/B testing has become one of the steps, tools and orientations that any product requires to optimise conversions, growth channels, marketing budgets and ultimately, profitability.
No sector is safe from the bottomless pit of assumptions and hunches. The ‘I feel this might work’ adage, or ‘we have always done it this way and got some really interesting results…’ quip is very cosily found in many a marketing department across industry and sectoral lines.
But what is A/B testing?
In very simple terms, split testing involves having a minimum of two variants of a digital asset. This could be two variants, the control and the B version of a web page, an application, subscription form, pricing structure, and more, and these two versions would be randomly pushed to users and then using different tools, the marketing and technical teams would analyse conversion rate based on a predefined goal.
The move to A/B testing involves knowing your users, their browsing patterns, their interests, click habits, and engagement points. Once these are analysed, one can come up with a suitable goal to track and test.
There are various examples of how to use A/B testing. The following are three examples of how to use A/B testing on a fictitious product called HelloVPN.
HelloVPN is a VPN product with a twist as it also contains an antivirus product. This affords the user a fuller protection and privacy while browsing the internet, at home or on public Wi-Fi. So far, so good.
The user base numbers are very encouraging, as the product rounds up to two million users daily from 40 countries. In terms of business model, it rests on a freemium model, whereby the free users get to access five countries, as proxy, randomly so as the user is not able to choose from which ‘country’ to browse from and no use of anti-virus. Fully paid solution provides 29 countries for proxy purposes, intelligent re-routing for better service and also full anti-virus solution in the background. Approximately one per cent of the base are actually paying for the service.
The first test is angle. The test here is to ascertain which angle converts the best – are users buying for acquiring VPN or anti-virus? One way (out of many) to test this is to have two sets of adverts: one promoting the anti-virus as primary, and another – variant – promoting the VPN/privacy as primary with anti-virus as secondary. These adverts need to be tied to related landing pages, with clear content that resonates with a clear call-to-action. Track conversion rate, and keep tweaking.
Testing involves knowing your users, their browsing patterns, their interests, click habits, and engagement points
The second test focuses on discounts. Pricing of products is extremely tricky. Typical pricing tricks involve the use of a nine – such as $19.99; or ending with five – such as $19.95. This to give the user the vibe that the spend is below $20 dollars.
One way of testing this – and what price converts the best – is to have an offer on the product, with varying degrees of discounts. Say the control price is $59.95. This price works in Canada, but not in Serbia, for instance (again, based on analysis and testing).
For Serbia, use the following two discount ratios: 45 per cent and 70 per cent. Tie these two discounts with the two angles explored above. Adapt two landing pages to reflect the discount, also given in per cent, and compare conversion rate.
As one final test, added to the above, consider the use of limited time offers (by adding a countdown clock, say offer expires in one minute) to one and not the other, for instance, would add a sense of urgency to get the user to convert more.
A/B testing implies a cultural shift for most organisations. For an organisation, commercial or otherwise (even NGOs can benefit from AB testing their donations page, for instance), to truly benefit from adopting tools, technologies, views and insights, the whole organisation needs to buy into the cultural shift the testing of assumptions requires. Growth, profitability, return on investment and ultimately survival of the business cannot be based on mere hunches; or false hypothesis, or incomplete information – but on data.
Data, crunching of numbers, testing of assumptions, agility, willingness to find new and interesting angles, together with a cross-departmental alignment to achieve such goals need to become the new norm, as your competitors are testing as you read this.