In our web era A/B tests have become the little, but important helpers of any online product. Unlike offline competitors, we have a unique chance to try different theories and conduct some experiments with minimum effort and funds required. So it would be a crime to deprive yourself of such a chance. But not all A/B tests are efficient. Today we will learn how to develop a good A/B test theory that will actually help your business.
Ask the right questions
A quick look at A/B test mechanics creates a false impression that it is extremely easy to create it. This is true and not true, since you not merely want to create an A/B test, but want to create an A/B test that brings viable results you can use for your marketing strategy.
And don’t forget that to make the results credible A/B test should last at least 30 days, so you definitely don’t want to waste your time here.
To make the test work for you, you should develop a good hypothesis to be tested. This is what usually happens and what should happen.
As you see, you should be more cunning and analytical here. Imagine yourself a medieval scientist creating a magic potion. Experiment with your ingredients, but don’t throw anything you see in the kettle.
To develop a good hypothesis you need to answer the following questions:
- What to test
- Why to test
Basically, it is all connected with figuring out cause (why) and consequence (what) of your online product behavior. You can create your hypothesis either way: find the consequence and then find out the reason of it or vice versa.
How to find WHAT to test?
Let’s start with the deductive approach by finding out results first and reason second. To do this you can use a number of analytical reports. The best one to be looking at are:
- Traffic Report: gives you a pretty good overview of how many people are at your website and what they are doing there
- Acquisition Report: good to figure out how people find your website
- Funnel Report: great for conversion analysis and getting idea whether you guide your customers to the desired result right.
- Device Type: make sure you look good for top-5 devices/browsers/OSs used
So if you see something that doesn’t meet the expected result, this might become a good reason to A/B it. But first make sure you develop a credible theory why you didn’t manage to hit the target. This is a goo example how you can build theories.
Follow the same pattern in your theories.
How to find WHY to test?
If you don’t like the first variant or just didn’t find anything worth your attention, it might be a good idea to go from reasons first. Here are a few ways you can do that:
- Visitor Behavior Analysis: yes, we talk about click rates, heat maps or even real user testing you organize. Check how users operate with your website and this might give you an idea of what problems they face up with.
- Surveys: if you don’t want to make guesses, just ask user in a quick survey. A good example of such surveys are pop-up windows asking ‘why the customer is leaving’ or select boxes with various reasons you see, when unsubscribe from newsletter. So if have no idea why user doesn’t follow the scenario you want, just ask them directly.
After you know the problem, brainstorm the solution to it and create an A/B test to see if it worked.
As you see, creating a good A/B test hypothesis is not that difficult: all you need is some research and a pinch of luck.
Are you doing A/B tests? Share what your insights are.
