Tag: Expérimentation

Article

Jul 28, 2014
4min read

Before you Start A/B Testing, Define your Roadmap

Anthony Brebion

Anthony Brebion

Share on

LinkedIn Facebook Twitter

What tests should I launch? In what order? How do I stop several tests from interfering with each other? What risks are involved and how do I minimise these? Before you begin, these are the questions to ask. Follow our advice to establish your roadmap and get results from your A/B tests.

Before you start testing, you must mark out your route. A roadmap is essential, not only for clearly defining your goals, priorities and risks, but also for setting out a timetable. With this, you can track the progress of your project, and keep all of your contributors informed.

Your roadmap should, in particular, consider the following 8 aspects:

Test names

Be sure to give your tests precise, explicit names (preferably the same as you use in your AB Tasty tool). For example, a short name such as “[HP]wordingCTA” is preferable to an entire phrase such as “Test on the wording of the CTA on the homepage”.

Test descriptions

So that all your contributors are in step with your testing procedure and can follow it as it evolves, write a short, explicit description. For example, “Replace CTA wording ‘Create an account’ with ‘Sign up now’” will allow anybody to understand the content and goals of the test.

Test priority level

It is vital to rank your tests in the order of their importance in order to decide the order in which you will launch them. It is up to you to gauge:

  • The expected benefits of the test
  • The technical difficulty of the test

After the results of your first few tests, you will be able to adjust your ranking to optimise the effectiveness of your tests.

Test range

It is also crucial to include in your roadmap the range of pages targeted by your test. This way, you will stop different tests from interfering with each other (when you have several tests taking place at the same time on the same page) or possible side effects. To guard against this, we advise you to cut your site into slices and to give each slice a different colour in your roadmap: for example, blue for the homepage, orange for category pages, green for product pages, yellow for the conversion funnel, etc.

Primary and secondary KPIs

For each test, you should define a primary key performance indicator (KPI) (associated with a macro conversion). This indicator, which is the main reason for creating the test, will allow you to evaluate its benefits. It might be the click-through rate from the button “Add to basket”, the number of signups to your newsletter, revenue generated, etc.

You should also define secondary KPIs (associated with microconversions), which will complement your analysis and allow you to better understand the results of your test. Examples might include time spent on the site, number of pages seen, bounce rate, and so on.

Resources required

Some tests might require:

  • Technical development
  • Ergonomic development
  • A specific launch date

These kinds of requirements must be specified in your roadmap.

Launch date and estimated end date

This information will make your team’s reports easier to read, and make it easier to plan potential future tests. In the meantime it will allow you to plan your testing activity precisely.

Possible side effects, Who to contact, Alerts

You should include a space for “notes” for each test, which will be useful in case of any problems. Here, you can write contacts, useful information, important things to remember, and so forth. This will save you a lot of time and worry.

Share on

LinkedIn Facebook Twitter

Subscribe to
our Newsletter

bloc Newsletter EN

We will process and store your personal data to respond to send you communications as described in our  Privacy Policy.

Article

Jul 17, 2014
5min read

Six Ways for Getting Started with A/B Testing with Low Traffic

Anthony Brebion

Anthony Brebion

Share on

LinkedIn Facebook Twitter

A/B testing is a key tool when it comes to optimizing your conversion rates. However, an effective A/B test campaign requires certain conditions and, in particular, a substantial level of traffic and conversions. Should you rule out A/B testing if your traffic is too low? The answer is no, you’ll be glad to hear! There are methods which can be implemented to make the most of an A/B test campaign, even when traffic is low. The main obstacle to setting up A/B tests with low traffic lies in statistical significance. If the test results are to be considered reliable, the reliability index needs to be over 95%. Below 95%, the data are not deemed reliable and drawing conclusions can be risky. In simple terms, the lower the traffic, the more time it takes to reach that level. The following techniques will help you reduce that time.

#1: be patient!

First of all, if your traffic does not exceed a few hundred visits per month, you are no doubt better off waiting. Focus on building an audience by applying the conventional traffic acquisition levers (production of interesting and potentially viral content, building your presence on the social networks, etc.). At this stage, take the opportunity to collect qualitative information on your visitors’ behaviour (their impressions, their feedback, etc.). As your traffic is low, this feedback can be analysed manually and can provide valuable avenues to be explored once you have acquired more traffic.

#2: get more traffic temporarily

If your traffic is low, tackle the problem at the source! Temporarily increasing traffic to the pages you want to test is a good way of obtaining reliable statistics more rapidly. The simplest way to do this is via a pay-per-performance advertising model (pay per click): sponsored Google AdWords links, Facebook Ads, sponsored news on LinkedIn, etc. This means investing financially, but it rapidly brings traffic. If you have a large community, set up an emailing campaign to draw traffic to the page you want to test. You should of course make sure you redirect your campaigns to a landing page that is close to the final conversion stage, rather than to your website’s home page.

#3: test the pages with the most traffic

Make the most of the traffic you already have by focusing efforts on the pages where you get the most traffic. By doing this, you will increase your chances of getting significant results faster. When you have more traffic across the whole website, you will then be able to test the other pages.

#4: limit the number of variations

The more variations you compare, the less each one of them will get traffic and the more time it will take to get a sufficient sample for each variation. Don’t create more than two variations in addition to the original version. Depending on your traffic, you may need to limit yourself to a single variation. If this out-performs, start a second test to compare its performance with the variation you had set aside. Also forget about multivariate tests (MVT) which are only designed for websites with very high traffic levels.

#5: change the conversion measurement criteria

The aim of an A/B test is usually to increase the number of conversions. But how do you define conversion? As a sale? In a test where traffic is low, it is preferable to select a criterion related to your main criterion but which occurs more often. Rather than a sale, target the downloading of a test version or the viewing of a demonstration video, for example. These conversion criteria are always directly linked to your main criterion (here, the sale), but are likely to occur more often and will therefore bring you results faster. You can also target interaction with your website, for example by recording a conversion when a visitor spends a certain amount of time on your website or visits a certain number of pages.

#6: test significant changes rather than small modifications

We often think that A/B testing is used to define the best color for a call-to-action or to optimise a title. This is just one facet of A/B testing but this kind of micro-optimization can only be applied where traffic is substantial. Where traffic is low, look at the bigger picture! Test changes likely to lead to a high increase in conversions, not the details seeking a 0.1% increase. Change the position of elements, rewrite entire titles and test two completely different versions of your page. If you opt for this latter option, do not waste time modifying your page in the AB Tasty editor but instead opt for the split testing solution. The principle is simple: you create a completely different design for your page, you host it on your own server and you use it as a variation. Web visitors are redirected to this page in a fully transparent manner. AB Tasty takes care of the traffic breakdown and collects statistics for each version, according to the parameters you have indicated, just like a conventional test.

Conclusion

Just because your website has low traffic, it does not mean you should forget about A/B testing – on the contrary! Set your goals and set you up your first tests.

Share on

LinkedIn Facebook Twitter