A Laymans Guide to A/B Testing
Every little detail matters when it comes to testing design changes. However, it can be pretty difficult to tell if a subtle change is really going to make that much of an impact. There are ways to test this theory out, thankfully. By using an A/B Test, you can determine if there is going to be a difference in the way users react between different options they are presented with.
What is A/B Testing?
Also known as split testing, A/B testing is when two versions of a website are compared in order to test which one is going to have the best performance. To compare the two different designs, you’ll label them A and B, and show them to similar visitors at one time. When you notice that one has a better conversion rate, that is the winning design. This can be very useful because even if there is only a difference of 1% on the purchases, that can be a very drastic amount over time, especially on websites that are very popular.
Of course, every website has its very own purpose. For example, an ecommerce website wants customers to continue buying from them, news websites want readers to click on an ad and sign up to become paid subscribers for the news. It is simple, a business always wants something from someone. This is called conversion, when a visitor to the website converts into something else. The conversion rate is the rate at which a website is actually able to convert visitors into what they want them to be. So an A/B test is going to measure which variant has a better conversion rate.
Why Perform A/B Testing?
It seems like a pretty straight-forward answer; you should try an A/B test if you want to improve the traffic you are seeing on your website. It doesn’t cost very much to increase your conversions, even if it does cost a lot to buy traffic for your website. This shows which option is a better choice, especially for the long-term goal. When you start A/B testing, you are starting on the journey to increasing the amount of conversions your website can do and create more sales which means more money for your business.
What Can Be Tested by an A/B Test?
Virtually every aspect of a website can be tested because everything has an impact on how a visitor perceives the content. Here’s a small list of what can be tested (not all inclusive): Headlines, paragraph text, call to action text, links, call to action button, testimonials, sub headlines, content near the fold, media mentions, social proof, awards and badges, as well as images can be tested. If you want to do a more advanced A/B test, you can compare pricing structures, free trial lengths, UX experiences, free or paid delivery, sales promotions, and navigation differences and see the different impacts.
SEO and A/B Tests
There’s a few important things you need to know about SEO and A/B Testing.
- Cloaking Is Not Allowed.
Cloaking is when one set of contents is showed to the humans and another set is shown to the Googlebot. This is against the Webmaster Guidelines, so you’re going to need to be careful. This means you want to ensure you aren’t deciding whether you are going to serve the test or which content variant you are going to be using based on user-agent. If you go against these rules, you can really hurt your rankings and could possibly even get removed from being able to be found from Google, which you really don’t want to happen.
- Don’t Use 301s, Use 302s.
The difference between 301s and 302s is that using a 302 is only going to temporarily redirect users to a different variation and a 301 is going to be permanent. This way, you can let the search engine know that the redirection is temporary and is only going to be used while the experiment is being conducted. This should keep the original URL in the index so it is not replaced by the test page you create. Of course, you can also use a JavaScript-based redirect.
- Don’t Run the Experiment Longer Than You Need to.
It will depend on several things for how long your test has to run for good results to be seen. From things like the conversion rates and the rate of traffic on your website, it is a good idea to have a reliable testing tool to let you know when enough data has been gathered to find out which variant should be used. Once the test has been finished, update your site with the winning content and take away anything to do with the test as quickly as you quick (alternate URLs and testing scripts for example).
How to Perform an A/B Test
The best way to perform an A/B test is to take a scientific approach. You want to follow a process in order to get the desired results.
- Analyze the Data on Your Website — You can use a website analytics tool, like Google Analytics. This will provide you with information about where your conversion rates are suffering. You’ll be able to tell which pages have the biggest bounce rate and be able to work on them to improve the issues.
- Watch How Users Behave — It is good if you use visitor behavior analysis tools. This way, you can see exactly what is keeping the visitors on your website from converting to what you need them to be. For example, if the button or link is not noticeable enough, it can cause people to simply overlook the opportunity to convert.
- Form a Hypothesis — You need to figure out what you think is going to happen. Form a hypothesis based on increasing your conversion from your chances. For instance, if you think that increase the size of a conversion button will make it more noticeable to visitors and thus, increase the conversion rate.
- Test the Hypothesis — You’re going to need to follow through with what you stated in your hypothesis. Use an A/B test to see if you were correct. Test the original version against your new version and see which one has a better rate of success.
- Record the Data and Form Conclusions — Look over the results your test produced. You should be able to see which kind of variation has the highest conversion rate. If there is a clear difference between the two, you’ll know which one you should use. If there is not a clear difference, repeat from step 3 and form a new hypothesis.
- Turn in Results — Once you have clear results, tell the Marketing, IT, and UX teams about the test results so the proper changes can be made.
Trying an A/B Test
It can be intimidating if you are first thinking about implementing an A/B test. You’ll want to have the Visual Website Optimizer code snippet in your website and create your variations using the WYSIWYG Visual Editor. Of course, you need to have your goals in mind before starting a new design. Then, all you have to do is start the test and keep an eye on your data!
