Is Running CRO Experiments Complicated?

It might be tempting to think that running a successful conversion rate optimization strategy is a significant investment; that you need a lot of special knowledge, whether it’s coding skills, experience with statistical analysis or project management; that it’s very time-consuming; and that, even though it produces results, it will give you a massive headache along the way.

While we won’t deny that if you want to scale up, get fancy with testing scenarios, or build up your team, you’ll need to put in the hours, we do want to dispel the myth that CRO always has to be ‘complicated’. For those just starting out, the barrier to entry can actually be quite low, depending on the tools and process you’re using. Here are a few reasons why:

Simple Experiments Can Lead to Huge Gain

One of the great things about CRO is that it’s often the simplest tests that lead to the best results. You don’t have to set up complex, multi-page scenarios or complicated multivariate tests to see significant ROI.

Online shop Gin Kiosk, for instance, ran a simple A/B test, predicated on the hypothesis that a ‘related products’ bar deserved the prime real estate on product pages, in the place of lengthy product reviews. Merely switching the placement of these two elements led to a revenue uplift of 27%. 


Gin foundry ab tasty
By simply moving the related products bar above the reviews, Gin Kiosk gained 27% more revenue.



In the example above, the Gin Kiosk team didn’t need to use any coding skills to make these changes and run their test, since they had a visual, WYSIWYG editor at their disposal. Platforms that have a built-in editor of this kind (and AB Tasty is naturally one of them) makes it extremely easy and intuitive to make visual changes to your website, without knowing how to code. Changing the color of a CTA, moving around blocks of text or images, updating a hero image…all of it is possible to do with zero tech skills.

Even better? To get the ball rolling with this kind of client-side testing, all you need is a one-time setup of a few lines of copy and pasted code.  Once this is done, you’re good to implement all the tests you want.


In a similar vein, having access to a library of virtually code-free widgets that you can use to increase audience engagement in a variety of ways is a huge time-saver. Adding a countdown banner for urgency or an exit intent pop-in to reduce bounce rate is a ten-minute investment with widgets, instead of an hours-long project necessitating help from your friendlist web developer colleague.

Built-In Statistical Analysis

Any reasonable experimentation platform will have easy-to-read reporting dashboards built into the tool. Nobody expects you, on top of your already busy schedule, to be running your own calculations to determine if your test results are actually viable. 

With clearly stipulated guidelines for reaching statistical reliability (in our case, we recommend letting a test run for two business cycles, with at least 5,000 recorded unique visitors and 300 conversions on the main goal) and solid statistical analysis (Bayesian, in our case), CRO professionals should be able to easily make sound decisions about whether or not to implement a new variation, without racking their brains over the math.

Our reporting interface makes it easy to understand when – and to what extent – a new variation has over or underperformed.


No Such Thing as a ‘Losing’ Test

Sure, when you run an experiment, you’ll be testing one variation against another to see which performs better. Certain versions will perform better according to your set KPIs, and will be declared the winners. In this sense, of course there are ‘winning’ and ‘losing’ tests.

But – at the risk of sounding trite – website optimizers do always learn something from a losing variation. These are valuable insights into your audience’s behavior that can be used to improve future experiments. 

Take the example of our client Pour de Bon, which runs an online marketplace for French farmers and consumers. The first few tests they ran produced negative results – and you’d be tempted to think they were a waste of time. But far from it! They learned about what works and what doesn’t for their audience, and adjusted each subsequent test based on the learnings from the last.  In a relatively short amount of time, they ended up hitting on their winning formula – and increased revenue by 25% thanks to their optimizations. 

Pour de bon
Pour de Bon found the perfect format of contextual, reassuring and in-the-moment messaging for their audience thanks to a progessive testing method.


All in all, Conversion Rate Optimization doesn’t need to be overwhelming, overly complex or out of reach for small teams or marketers without coding skills. And once you get the hang of it and run a few successful tests, you can prove the value to those who hold the pursestrings, build out the team and start scaling up. Resources permitting, it might be time to start playing around with server-side testing, or ramping up the number or complexity of your tests – but only when you’re ready. But one thing’s for sure – CRO doesn’t have to be complicated!

CRO Myth Busters is a mini series for conversion rate optimization professionals. We take a quick look at commonly held CRO beliefs and determine if they’re true, sometimes true, or simply CRO myths!

Share on linkedin
Share on Linkedin
Share on facebook
Share on Facebook
Share on twitter
Share on Twitter

You might also like...

Don't throw away hard-earned traffic.
Make your website work overtime - so you don't have to.
click-through rate
conversion rate
average order value
Trusted by
Request a demo