Share on

LinkedIn Facebook Twitter

Breaking news: according to CopyBlogger, 80% of all readers never make it past the headline.

If you read this, you’re among our happy 20% and you shall not be disappointed.

The truth is: it’s a pretty serious issue for all publishers.

Similarly, the Washington Post reported that 6 in 10 Americans acknowledge that they don’t read past the headlines regarding any type of news.

So, should we just stop writing?

Obviously not.

In 2018, the written content is still one of the most consumed media (in competition with videos) and remains a powerful tool to:

• Build brand awareness
• Generate B2B Leads
• Report news
• Drive sales
• Grow your audience
• Rank on search engines

Knowing that most readers won’t spend more than 15 seconds reading an average article (source: Buffer), crafting powerful and catchy headlines has never been more important to ensure that your audience will stick around for a while and that you don’t produce content in vain.

But how do you make sure that your headlines really work?

It’s simple: you need to run some headline A/B testing.

What is headline testing?

Based on your objectives, headline testing can be used to track several metrics such as:

• Number of shares
• Average Reading Time
• Click-Through-Rate
• Bounce Rate
• Conversions

How to conduct headline testing

Headline testing requires you to define a title as the “control version” in order to compare it with one or more variant.

While choosing the number of variants, bear in mind that the more variants you wish to test, the larger sample you will need in order to obtain statistically relevant results.

Once you’ve chosen your variants, you will use an A/B testing tool to run your tests and see which headline outperforms the others.

Typically, an A/B testing tool will send a percentage of your page’s traffic to each variant until it identifies a winner.

From there, the tool will allocate 100% of the traffic to the “winner” in order to maximize your page’s performance.

Sound good?

Let’s see how to come up with brilliant headline ideas that you will be able to A/B test later on.

How to brainstorm headline ideas

Headlines come in many forms depending on whether you’re writing an article, a landing page or even a product description.

Given this variety of headlines, we’ll try to help you craft various headlines through general guidelines to meet your business objectives.

Include real data and numbers

Numbers act like candies for the brain: we just love to see them because they give us facts and figures to work on.

In 2013, Conductor came up with a study that showed the impact of adding numbers to your headlines: it appears that readers do prefer headlines that include numbers.

Craft a strong value proposition

Creating a value proposition for your readers means that you need to work on including a real benefit inside your headline.

Working on your value proposition is the cornerstone of every headline creation process: it helps you address your core audience while promising something in exchange for their attention.

Depending on the content you’re working on, crafting your value proposition is a process that basically sells your content: it will determine whether or not your potential readers will click on your content.

Here are some formulations commonly used to craft a strong value proposition:

• Recipes for success
• Expert opinions
• Special offers and discounts
• Tips and advice
• Guides, ebooks
• Facts, studies
• Ideas, strategies

Trigger your readers’ curiosity

Capturing your readers’ attention is no easy task given the average level of online competition that most publishers encounter.

In order to grab your visitors’ attention from the beginning, try to avoid headlines that can easily be answered by “Yes” or “No”.

“Yes and No” headlines are dangerous because they force your visitors to form an opinion about your question or your statement; which will eventually lead to a significant share of visitors choosing not to click.

Here’s a list of formulations used to trigger curiosity:

• “How to …”
• “The 7 facts you didn’t know about …”
• “How [insert_name] managed to [action] in [days]”
• “The Complete Guide to …”
• “What every [target] should know about [subject]”

Watch your competition

There’s no secret for marketing success: practice makes perfect.

Because most businesses typically have dozens of competitors, you should pay attention to your competitors’ headline formulations.

From there, try to identify general trends and success formulas that you could apply to your own content.

Ideas for effective headlines from the competition can be found in:

• Newsletters
• Websites pages and landing pages
• Product descriptions
• Ebooks
• SERPs (Search Engine Result Pages)

Keep them simple and clear

Online visitors and shoppers are over-exposed to marketing messages all day long.

Knowing this, it can be clever to keep your headlines short, simple and clear in order to deliver straightforward information to your potential readers.

Because marketers are always searching for new magic formulas, they sometimes come up with complex, tricky formulations that you should avoid.

Use a headline analyzer

Headlines analyzers are online tools that score your headlines based on a certain number of parameters.

Typically, these tools will grade your headlines on a 100 scale in order to help you craft catchier, better headlines.

They often measure the length and analyze your headline’s structure to determine optimal word order, keyword use, and formulation.

Here are 2 free tools you can use to analyze your headlines:

• CoSchedule
• ShareThrough

We’ve analyzed our own headline to see what type of results we would get.

Key Takeaway: our headline “How to Effectively A/B Test your Content Headlines” scored a reassuring 72/100 because it contains a power word “effectively” and an emotional element that triggers curiosity “How to…”.

The tool even identified our main keywords, which is a good starter for search engine optimization.

Run A/B tests and compare results

As you know, headline testing can bring tremendous benefits to your key metrics such as page views, CTR and conversions.

To prove this point, Priceonomics came with an analysis that showed a 33% improvement on pageviews following headline testing: a major improvement that could drastically improve the way visitors behave on your website.

Now that you’ve come across our best practices for headline creation, it’s high time you start testing your own headlines variations to find out the most effective ones.

In order to do so, here’s a little checklist you can follow:

1. Use our A/B Testing tool to set up your experimental environment
2. Our WYSIWYG editor makes it easy to test headlines
3. Start brainstorming headline ideas and formulate hypotheses
4. Try to run some headline ideas through CoSchedule to measure your chances of success.
5. Run your tests and collect results
6. Measure your tests results and track important KPIs to monitor any change

Did you like this article? Feel free to share and check out our other in-depth articles on how to optimize your website, ecommerce and digital marketing.

Share on

LinkedIn Facebook Twitter

Share on

LinkedIn Facebook Twitter

A/B testing is an effective way to improve your site’s user experience and its ability to convert users to clients.

While changes made to your site may impact your user’s behavior, they are also seen by search engine crawlers, especially Google. The latter is perfectly capable of interpreting JavaScript, the scripting technology behind a lot of A/B tests.

This post is a good opportunity to review A/B testing best practices for SEO and help you do what’s best when it comes to optimizing conversions, without jeopardizing your rankings and web traffic.

General SEO recommendations

To start, let’s review some general recommendations from Google.

Google completely accepts A/B testing and even encourages it if it’s geared towards improving user experience. Google also offers its own client-side A/B testing tool (Google Optimize) that uses JavaScript to manipulate the DOM (Document Object Model) to create page variations.

On its blog, Google shares rules to be respected so that its algorithms do not penalize your site. The main rule concerns opening your test to the search engine’s robots, who must navigate on the same version of your pages as your visitors.

So, one of the first best practices for SEO is to not exclude Google’s bot from your A/B tests. Even if your A/B testing solution offers some advanced user-targeting capabilities, like user-agent detection, do not use them to exclude Googlebot.

It is also recommended that you do not display pages that are too different from one another to your users. For one, it will be more difficult to identify which element(s) had a greater impact on the conversion rate. Second, Google may consider the two versions to be different and to interpret that action as a manipulation attempt. Losing ranking may result or, worst case scenario, your site may be completely removed.

Depending on your objectives, the A/B testing setup may differ and each way of doing things can have an impact on SEO.

Best practices for A/B tests with URL redirects

A/B testing using URL redirects, also known as split testing, is one of these methods. Instead of using a WYSIWYG (What You See Is What You Get) editor to design your variation, you redirect users to a completely separate page, often hosted on your site, that has its own URL. Using this method is justified if you have a lot of changes to make on your page; for example, when you want to test a different design or another landing page concept.

This use case is the most prone to error and can have a dramatic impact on your search engine ranking, namely your original page being removed from the Google index, and replaced by your variant page. To avoid this, remember the following points:

• Never block Google’s bots via your site’s robots.txt file with the Disallow instruction or by adding the noindex command on your alternate pages. The first prevents bots from reading the content of targeted pages, whereas the latter prevents them from adding the pages to Google’s index. It’s a common error, as the site publisher is afraid that the alternate version will appear in results. If you respect the following instructions, there is no reason for your alternate version to “rank” instead of your original version.
• Place a canonical attribute on the variant page and set the value to the original page. This tells Google the original page is the one it must take into account and offer to internet users. Search engine bots will understand that page B has no added value compared to A, which is the only version to be indexed. In the case of a test on a set of pages (e.g. you want to test 2 product page formats across your catalog), you must set up this matching for each page.
• Redirect visitors via a 302 or JavaScript redirection, both of which Google interprets as temporary redirects. In other words, the search engine considers it to be a temporary modification of your site and does not modify its index accordingly.
• When a redirect test is completed, you must put into production the changes that have been shown to be useful. The original page A is then modified to include the new elements that foster conversion. Page B, meanwhile, can either be redirected to page A with a 301 (permanent) or 302 (temporary, if the page will be used for other tests) redirection.

Best practices for standard A/B tests

Applying a JavaScript overlay is by far the most common way to conduct A/B tests. In this case, your variants are no more or less than changes applied on the fly when the page loads into the user’s browser. The A/B testing solution manages the whole process from the JavaScript code interpretation of changes you made via a graphics editor, up to data collection, by randomly assigning users to one of the variants and respecting this assignment throughout the test. In this case, your URLs do not change and changes only occur in the client browser (Chrome, Firefox, Internet Explorer, etc.).

This type of A/B test does not harm your SEO efforts. While Google is perfectly capable of understanding JavaScript code, these changes will not be a problem if you do not try to trick it by showing it an initial content that is very different from that presented to users. Therefore, make sure that:

• The number of elements called by the overlay is limited given the overall page and that the test does not overhaul the page’s structure or content.
• The overlays do not delete or hide elements that are important for the page’s ranking and improve its legitimacy in the eyes of Google (text areas, title, images, internal links, etc.).
• Only run the experiment as long as necessary. Google knows that the time required for a test will vary depending on how much traffic the tested page gets, but says you should avoid running tests for an unnecessarily long time as they may interpret this as an attempt to deceive, especially if you’re serving one content variant to a large percentage of your users.

What about mobile SEO?

Using your A/B testing solution to improve the user journey on mobile devices is a use case that we sometimes encounter. This is a particularly sensitive point for SEO since Google is rolling out its Mobile First Indexing.

Until now, Google’s ranking algorithm was based primarily on the content of a site’s desktop version to position it in both desktop and mobile search results. With the Mobile First Indexing algorithm, Google is switching this logic around: the search engine will now use the mobile page’s content as a ranking signal rather than the desktop version, no matter what the device.

Therefore, it’s particularly important to not remove from mobile navigation – for UX reasons – elements that are vital to SEO, like, for example, removing page-top content that takes up too much space on a smartphone.

Can personalization impact your SEO?

Some A/B testing tools also offer user personalization capabilities. AB Tasty, for example, helps you boost user engagement via custom scenarios. Depending on your visitors’ profile or their journeys on your website, you can easily offer them messages or a personalized browsing experience that is more likely to help them convert.

Can these practices have an impact on your SEO? Like for A/B tests using JavaScript, impact from SEO is limited but some special cases should be taken into consideration.

For instance, highlighting customized content with an interstitial (pop-in) presents a challenge in terms of SEO, notably on mobile. Since January 2017, Google considers it to be harmful to the user experience since the page’s content is not easily accessible. So personalized interstitials must be adjusted to Google’s expectations. Otherwise, you take the risk of seeing your site lose ranking and the resulting traffic.

Note that Google seems to tolerate legal interstitials that take up a majority of the screen (cookie information, age verification, etc.) for which there is no SEO impact.

To learn more, download your free copy of our A/B testing 101 ebook.

Share on

LinkedIn Facebook Twitter

Share on

LinkedIn Facebook Twitter

Wait! New to multivariate testing? If so, we recommend you first read our article, Multivariate Testing: All you need to know about multivariate testing

During an A/B test, you must only modify one element at a time (for example, the wording of an action button) to be able to determine the impact. If you simultaneously change this button’s wording and color (for example, a blue “Buy” button vs. red “Purchase” button) and see an improvement, how do you know which of the wording or the color changes really contributed to this result? The contribution of one may be negligible, or the two may have contributed equally.

The benefits of multivariate tests

A multivariate test aims to answer this question. With this type of experiment, you test a hypothesis for which several variables are modified and determine which is the best combination of all possible ones. If you change two variables and each has three possibilities, you have nine combinations between which to decide (number of variants of the first variable X number of possibilities of the second).

Multivariate testing has three benefits:

• avoid having to conduct several A/B tests one after the other, saving you time since we can look at a multivariate test as several A/B tests conducted simultaneously on the same page,
• determine the contribution of each variable to the measured gains,
• measure the interaction effects between several supposedly independent elements (for example, page title and visual illustration).

Types of multivariate tests

There are two major methods for conducting multivariate tests:

• “Full Factorial“: this is the method that is usually referred to as multivariate testing. With this method, all combinations of variables are designed and tested on an equal part of your traffic. If you test two variants for one element and three variants for another, each of the six combinations will be assigned to 16.66% of your traffic.
• “Fractional Factorial“: as its name suggests, only a fraction of all combinations is actually subjected to your traffic. The conversion rate of untested combinations is statistically deduced based on that of those actually tested. This method has the disadvantage of being less precise but requires less traffic.

While multivariate testing seems to be a panacea, you should be aware of several limitations that, in practice, limit its appeal in specific cases.

Limits of multivariate tests

The first limit concerns the volume of visitors to subject to your test to obtain usable results. By multiplying the number of variables and possibilities tested, you can quickly reach a significant number of combinations. The sample assigned to each combination will be reduced mechanically. Where, for a typical A/B test, you are allocating 50% of your traffic to the original and the variant, you are only allocating 5, 10, or 15% of your traffic to each combination in a multivariate test. In practice, this often translates into longer tests and an inability to achieve the statistical reliability needed for decision-making. This is especially true if you are testing deeper pages with lower traffic, which is often the case if you test command tunnels or landing pages for traffic acquisition campaigns.

The second disadvantage is related to the way the multivariate test is brought into consideration. In some cases, it is the result of an admission of weakness: users do not know exactly what to test and think that by testing several things at once, they will find something to use. We often find small modifications at work in these tests. A/B testing, on the other hand, imposes greater rigor and better identification of test hypotheses, which generally leads to more creative tests supported by data and with better results.

The third disadvantage is related to complexity. Conducting an A/B test is much simpler, especially in the analysis of the results. You do not need to perform complex mental gymnastics to try to understand why one element interacts positively with another in one case and not in another. Keeping a process simple and fast to execute allows you to be more confident and quickly iterate your optimization ideas.

Conclusion

While multivariate tests are attractive on paper, note that carrying out tests for too long only to obtain weak statistical reliability can make them a less attractive option in some cases. In order to obtain actionable results that can be quickly identified, in 90% of cases, it is better to stick to traditional A/B tests (or A/B/C/D). This is the ratio found among our customers, including those with an audience of hundreds of thousands or even millions of visitors. The remaining 10% of tests are better reserved for fine-tuning when you are comfortable with the testing practice, have achieved significant gains through your A/B tests, and are looking to exceed certain conversion thresholds or to gain a few increments.

Finally, it is always helpful to remember that, more than the type of test (A/B vs. multivariate), it is the quality of your hypotheses – and by extension that of your work of understanding conversion problems – which will be the determining factor in getting boosts and convincing results from your testing activity.

Share on

LinkedIn Facebook Twitter

Share on

LinkedIn Facebook Twitter

A/A testing is little known and subject to strong discussions on its usefulness, but it brings added value for those who are looking to integrate an A/B testing software with rigor and precision.

But before we begin…

What is A/A testing?

A/A testing is a derivative of A/B testing (check out A/B testing definition). However, instead of comparing two different versions (of your homepage, for example), here we compare two identical versions.

Two identical versions? Yes!

The main purpose of A/A testing is simple: verify that the A/B testing solution has been correctly configured and is effective.

We use A/A testing in three cases:

• To check that an A/B testing tool is accurate
• To set a conversion rate as reference for future tests
• To decide on an optimal sample size for A/B tests

Checking the accuracy of the A/B Testing tool

When performing an A/A test, we compare two strictly identical versions of the same page.

Of course, the purpose of an A/A test is to display similar values in terms of conversion. The idea here is to prove that the test solution is effective.

Logically, we will organize an A/A test when we set up a new A/B test solution or when we go from one solution to another.

However, sometimes a “winner” is declared on two identical versions. Therefore, we must seek to understand “why” and this is the benefit of A/A testing.

• The test may not have been conducted correctly
• The tool may not have been configured correctly
• The A/B testing solution may not be effective.

Setting a reference conversion rate

Let’s imagine that you want to set up a series of A/B tests on your homepage. You set up the solution but a problem arises: you do not know to which conversion rate to compare the different versions to.

In this case, an A/A Test will help you find the “reference” conversion rate for your future A/B tests.

For example, you begin an A/A Test on your homepage where the goal is to fill out a contact form. When comparing the results, you get nearly identical results (and this is normal): 5.01% and 5.05% conversions. You can now use this data with the certainty that it truly represents your conversion rate and activate your A/B tests to try to exceed this rate. If your A/B tests tell you that a “better” variant achieves 5.05% conversion, it actually means that there is no progress.

Finding a sample size for future tests

The problem in comparing two similar versions is the “luck” factor.

Since the tests are formulated on a statistical basis, there is a margin of error that can influence the results of your A/B testing campaigns.

It’s no secret how to reduce this margin of error: you have to increase the sample size to reduce the risk that random factors (so-called “luck”) skew the results.

By performing an A/A test, you can “see” at what sample size the test solution comes closest to “perfect equality” between your identical versions.

In short, an A/A test allows you to find the sample size at which the “luck” factor is minimized; you can then use that sample size for your future A/B tests. That said, A/B tests generally require a smaller sample size.

A/A testing: a waste of time?

The question is hotly debated in the field of A/B Testing: should we take the time to do an A/A test before doing an A/B test?

And that is the heart of the issue: time.

In fact, performing A/A tests takes time, considerably more time than A/B tests since the volume of traffic needed to prove that the two “identical variants” lead to the same conversion rate is significant.

The problem, according to ConversionXL, is that A/A testing is time-consuming and encroaches on traffic that could be used to conduct “real tests,” i.e., those intended to compare two variants.

Finally, A/A testing is much easier to set up on high traffic sites.

The idea is that if you run a site that is being launched or has low traffic, it is useless to waste your time doing an A/A test: focus instead on optimizing your purchase tunnel or on your Customer Lifetime Value: the results will be much more convincing and, especially, must more interesting.

An interesting alternative: data comparison

To check the accuracy of your A/B Testing solution, there is another way that is easy to set up. To do this, your A/B Testing solution needs to integrate another source of analytic data.

By doing this, you can compare the data and see if it points to the same result: it’s another way to check the effectiveness of your test solution.

If you notice significant differences in data between the two sources, you know that one of them is:

• Either poorly configured,
• Or ineffective and must be changed.

Did you like this article? We would love to talk to you more about it.

Share on

LinkedIn Facebook Twitter

Share on

LinkedIn Facebook Twitter

Flickering, also called FOOC (Flash of Original Content) is when an original page is briefly displayed before the alternative appears during an A/B test. This happens due to the time it takes for the browser to process modifications. There is no miracle fix to this problem, and those claiming to be quick fixes have limited effectiveness. The good news is that there are several best practices to accelerate the application of your modifications, effectively masking the flickering effect.

What is flickering, exactly?

Although you may have never heard of flickering, you have undoubtedly experienced it without even knowing: a test page loads and, after a few milliseconds, your modifications show up. In the blink of an eye, you’ve seen two versions of your page—the old and the new. The result is poor user experience, not to mention that your users now know your test is just that: a test.

Flickering is caused by the basic client-side operation of A/B testing solutions that apply a JavaScript overlayer during page loading to ensure elements are modified. In most cases, you will not notice it at all, but if your site takes a while to load or relies on intensive external resources, your modifications can take time to be applied, giving way to a previously unnoticeable flickering.

Is there a miracle cure for flickering?

Some providers claim to use innovative techniques that get rid of flickering. Beware, however, that although the techniques they use are commonplace and available to anyone, they present a certain number of technical limits. By analyzing market leaders’ documentation, it is also clear that such “miracle” methods are only implemented as a last resort, when no other options have worked for a lasting period of time. This is because flickering can be different for each site and depends a great deal on initial performance.

So how does the method work? For starters, displayed content is temporarily masked using CSS properties such as visibility: hidden or display: none for the body element. This property masks page content as quickly as possible (since the solution’s tag is located in the page’s <head> element), before redisplaying it again once the modifications have had enough time to be applied. This effectively eliminates the “before/after” flicker effect, but replaces it with a “blank page/after” effect.

The risk of using such a method is that if the page encounters any loading problems or there are implementation problems, users might end up with a blank page for a few seconds, or they could even be stuck with a blank screen with nowhere to click. Another drawback of this solution is that it gives off the impression that site performance is slow. That’s why it is important to ensure that rendering is not delayed for more than a few milliseconds at most—just enough for the modifications to be applied. And of course, for valid results, you’ll need to apply this delayed rendering to a control group to prevent bias in your analysis of behaviors linked to the various rendering speeds.

So there you have it. If your modifications take time to apply, you won’t want a blank page to greet your users. When it comes down to it, you should always adhere to the best practices listed below. Among other things, they aim to ensure modifications are applied at an accelerated rate.

That’s why we here at AB Tasty don’t recommend the above method for most of our users and why we don’t suggest it by default. Nonetheless, it is easy to implement with just a few lines of JavaScript.

How can flickering be limited?

If you don’t want to use the aforementioned method, what are your options? Here are some best practices you can use to reduce flickering and maybe even eliminate it:

• Optimize your site’s loading time by all means possible: page caching, compression, image optimization, CDNs, parallel query processing with the HTTP/2 protocol, etc.
• Place the A/B testing solution tag as high as possible in the source code, inside the <head> element and before intensive external resources (e.g. web fonts, JavaScript libraries, etc.) are called.
• Use the synchronous version of the AB Tasty script, since the asynchronous version increases flickering odds
• Don’t use a tag manager to call your solution’s tags (e.g. Google Tag Manager). This might not be as convenient, but you’ll have an easier handle on your tag’s firing priority.
• Do not insert a jQuery library in the tag if your site provider already uses one. Most client-side A/B testing solutions use jQuery. AB Tasty nonetheless makes it so you don’t have to integrate the infamous JavaScript framework if you already use it elsewhere, so you can cross a few kb off your transfer list.
• Reduce the size of your solution’s script by removing inactive tests. Some solutions include the entirety of your tests in their script, whether they are suspended or in draft mode. AB Tasty includes only active tests by default. If, however, you have a high number of ongoing customizations, it might be appropriate to make them permanently operational and deactivate them on AB Tasty.
• Pay attention to the nature of modifications. Adding several hundred lines of code to obtain your modification will cause more flickering than a minor change to CSS styles or wording.
• Rely as much as possible on style sheets. It is usually possible to obtain the desired visual effect using style sheets. For example, you can simply implement a JavaScript instruction that adds a CSS class to an element, letting the class modify its aspect, rather than writing lines of script to manipulate its style.
• Optimize your modified code. When fiddling around with the WYSIWYG editor to try and implement your changes, you may add several unnecessary JavaScript instructions. Quickly analyze the generated code in the “Edit Code” tab and optimize it by rearranging it or removing needless parts.
• Ensure that your chosen solution uses one (or many) CDNs so the script containing your modifications can be loaded as quickly as possible, wherever your user is located.
• For advanced users: Add jQuery selectors to the cache as objects so they don’t need to be reorganized in the DOM multiple times. You can also make modifications in JavaScript rather than in jQuery, particularly when dealing with elements by class or ID.
• Use redirect tests where possible and if useful after an evaluation of the relation between the modification’s nature and the time required to put the test into place.

If you still see flickering after performing these optimizations, you may want to use the content-masking technique detailed above. If you’re not comfortable doing this alone, contact our support team.

Share on

LinkedIn Facebook Twitter

Share on

LinkedIn Facebook Twitter

AB Tasty’s note: This is a guest post by Umesh Kumar, digital marketing evangelist at Designhill.

A/B Testing isn’t a rocket science to understand and implement. It is just about testing two different versions of a page on your site to see which one attracts more audience. More than anything else, this test helps you know and understand your consumers better. After you run an A/B test, you will find that a few more have added in your earlier list of customers.

It surely is one of the best ways to improve your conversion rates. In fact, an article published in CrazyEgg.com reveals that using correct testing methods can increase conversion rates up to 300 percent. But it’s shocking that still, the majority of marketers choose not to use A/B test experiments. Don’t believe us? Check out the following stats:

• For most businesses, conversion rates range from just 1 to 3%. (Source: truconversion.com)
• Most companies spend $92 to acquire customers, but only $1 to convert them. (Source: tweakyourbiz.com)
• Only 22% of businesses are satisfied with their conversion (Source: Contentsquare.com)
• 44% of companies use A/B testing when trying to optimize conversion rates. (Source: Unbounce.com)
• 85% of marketers plan to focus more on conversion rate optimization this year. (Source: Truconversion.com)

Given these stats, it’s no surprise that many marketers steer clear of A/B testing for optimizing their site. But, how exactly can you optimize your conversions with A/B Testing? The answer is simple! Why not do what smart marketers do? Learn lessons from companies that have emerged as shining examples of A/B testing genius.

No matter what is the nature of your business, there is no harm in taking a step back and learning from others’ achievements. To help you, we have listed 5 classic case studies that will provide you with interesting test hypotheses and give you an insight on what and how visitors think. You can learn a lot from these case studies and use the learning to take on the conversion challenges in your way to success headfirst. These examples are quite simple to implement with any A/B testing tool.

Case Study 1: Conversions improve by 11.5% by Adding FAQs, Statistics and Social Proof on Websites

Test Carried Out By Kiva.org, an innovative non-profit organization, allows people to lend money via the Internet to low-income entrepreneurs and students across countries. Kiva conducted an A/B Test as they wanted to increase the number of donations from first-time visitors to their landing page.

Hypothesis: Giving more information to visitors coming to Kiva’s landing page will help boost the number of donors.

Result: Donations increased by 11.5% after adding an information box at the bottom of the landing page.

Version A – original: (left)

Version B: Addition of information box (FAQ, social proof & statistics)

What You Can Learn from This Test?

Ensure that your landing page is designed in such a way that it answers all questions that a visitor may have. In this case, the information box at the bottom of the page helped the organization explain about themselves and their services, providing statistics. The information increased their trustworthiness and credibility as a site.

Case Study 2: 2.57% increase in Open Rates 5.84% Higher Click-through Rate (CTR) by Changing the Subject Line of an Email

Test Carried Out By Designhill.com, one of the fastest-growing peer-to-peer crowdsourcing platforms that connect graphic artists with design seekers. They did an email blast a few days before Christmas to promote its content and increase the click-through rate.

Hypothesis: Just mentioning the title of the blog in the subject line of the email would get the majority of click-through rather than requesting recipients to review the post with the blog’s title.

Just writing “Top 10 Off-Page SEO Strategies for Startups in 2015” in the subject line of the email would get the majority of click-through rather than writing “Check out My Recent Post – Top 10 Off-Page SEO Strategies for Startups in 2015”.

Result: The company was able to score 5.84% higher CTR and 2.57% higher open rate by including just the title of the blog in the subject line.

What You Can Learn From This Test:

Your subject line is the first thing the recipient of your email sees. Therefore, your subject line must have the power to entice the readers to open the mail and know more about your products or services. Because after all, it doesn’t really matter what your offer is if it is not opened by your readers. Therefore, choose your words wisely as they will have a higher impact on open rates and click-through. But it’s not only important to ensure great subject lines, but you must also ensure that your logo design is laid out in a way that vital information pop-up. For example, your logo design and contact details must be easily locatable. For another, CTAs and other links must be out of clutter. Read our Beginner’s Guide to A/B Testing your Emails.

Case Study 3:  49% Increase in CTR by Adding Text in the Call-to-Action Button

Test carried out by Fab, an online community whose members can buy and sell apparel, home goods, accessories, collectibles, etc.

Hypothesis: Making the “Add to Cart” button clearer (by adding text) will lead to an increase in the number of people adding items to their shopping carts.

Result: There was an increase of 49% in CTR over the original after the text “Add to Cart” was included in the CTA button rather than just an image or symbol.

In the following image, you’ll see that the original design (on the far left) features a small shopping cart with a “+” sign and no text. The two versions (middle and right) added text-based designs. Version A helped increase cart adds by 49% over the original.

What You Can Learn From this Test:

Text connects better with visitors than images or symbols which may confuse them. Therefore, try and have a direct and clear CTA which will help consumers know of their actions.

It makes no sense to have a CTA that your visitors don’t understand. They don’t get to know what the button does actually.

Case Study 4: Conversion Rate Improved by 7.46% by Adding an Image Rather than a Blank Background. 

Test Carried Out By A company who wanted to check if customers get attracted to a blank background or one with pictures more.

Hypothesis: Using a photo in the background will lead to more conversions than a blank background.

Result: The conversion rate of the background with a photo was 25.14% as compared to 7.68% for the one without a photo.

What you can learn from this test:

You must have heard that a picture is worth a thousand words. People love visuals and there could be no other better place than your site to impress them with the pictures of your products. Having an image of your product or service in the background of your site can drive conversions as they get to see (visualize) what they’ll be getting. Images hold the power to grab the attention of your audience and turn them into customers.

Case Study 5: Leads Increase by 256% after Having a Mobile-Optimized Landing Page

Test Carried Out By Rasmussen College, a for-profit private college and Public Benefit Corporation, who wanted to increase leads from Pay-Per-Click traffic on their mobile site.

Hypothesis: Creating a new mobile-friendly website, featuring a click-through menu, will improve conversions.

Result: Conversions increased by 256% after a new mobile-only (mobile responsive) site was made.

What you can learn from this test:

Hammad Akbar, founder of TruConversion.com says that “Unpleasant mobile experiences and a lack of mobile-friendliness makes 48% users believe that the company doesn’t care about their business.” A mobile-responsive website enhances the browsing experience of the site visitors. It is essential if you don’t want to lose customers just because your site took time to load. Keep the design of your site simple with only the basic information on the first page. Try and find different way of improving mobile navigation such as having a drop-down menu.

Conclusion

We hope that after reading this post, you are inspired to hold some amazing A/B tests on your own site. It is actually exciting and amazing to see what your customers like or dislike. But don’t forget that these tests are mere a guiding tool and nothing can replace your own tests and judgments about your visitors as well as the site. Remember, there is always a scope for improvement.

So, happy testing!

Download our guide to learn all there is to know about A/B testing!

Author Bio: Umesh Kumar is a digital marketing evangelist who works with Designhill, the fastest-growing custom design marketplace, to develop and execute their marketing strategies. He started his online marketing career in 2008 and is actively engaged within internet business modeling, website development, social networks, lead generation, search engine optimization, and search engine marketing. In addition, he loves blogging and shares his expertise about tips, tricks, and upcoming trends in the world of digital marketing. Get in touch Facebook | Twitter | Google+

Share on

LinkedIn Facebook Twitter