Tag: EN

Article

Jun 20, 2023

9min read

A/B, Split or Multivariate Test: How to Choose the Right One

Anthony Brebion

In the fast-paced world of digital marketing, settling for anything less than the best user experience is simply not an option.

Every marketing strategy has room for improvement and unlocking more comes from recognizing hidden opportunities.

With analytics data and a little bit of creativity, you can uncover some valuable insights on how to optimize your conversion rate on your website or campaign landing pages. However, achieving structured and streamlined data from your assumptions requires diligent testing.

Marketing professionals have steadily used different testing methodologies such as A/B testing, split testing, multivariate testing and multipage testing to increase conversion rates and enhance digital performance.

Experimenting and testing are essential as they eliminate opinions and bias from the decision-making process, ensuring data-driven decisions.

With the availability of many diverse testing options, it can be challenging to find your starting point. In this article, we’ll dive into the specifics of different forms of testing to help you navigate this testing landscape.

What is A/B testing?

flowers-366155_1280

A/B testing is a method of website optimization where you are comparing two versions of the same page: variation A and variation B. For the comparison, it’s common to look at the conversion rates and metrics that matter to your business (clicks, page views, purchases, etc) while using live traffic.

It’s also possible to do an A/B/C/D test when you need to test more than two content variations. The A/B/C/D method will allow you to test three or more variations of a page at once instead of testing only one variation against the control version of the page.

When to use A/B tests?

A/B tests are an excellent method to test radically different ideas for conversion rate optimization or small changes on a page.

A/B testing is the right method to choose if you don’t have a large amount of traffic to your site. Why is this? A/B tests can deliver reliable data very quickly, without a large amount of traffic. This is a great approach to experimentation to maximize test time to achieve fast results.

If you have a high-traffic website, you can evaluate the performance of a much broader set of variations. However, there is no need to test 20 different variations of the same element, even if you have adequate traffic. It’s important to have a strategy when approaching experimentation.

Want to start testing? AB Tasty is the best-in-class experience optimization platform that empowers you to create a richer digital experience – fast. From experimentation to personalization, this solution can help you activate and engage your audience to boost your conversions.

Split testing vs A/B testing

A/B tests and split tests are essentially the same concept.

“A/B” refers to the two variations of the same URL where changes are made “live” using Javascript on the original page. SaaS tools that provide you with a visual editor, like AB Tasty, allow you to create these changes quickly without technical knowledge.

Meanwhile, “split” refers to the traffic redirection towards one variation or another, each hosted on its own URL and fully redesigned in the code.

You can consider A/B tests to work the same as split tests.

The variation page may differ in many aspects depending on the testing hypothesis you put forth and your industry goals (layout, design, pictures, headlines, sub-headlines, calls to action, offers, button colors, etc.).

In any case, the number of conversions on each page’s variation is compared once each variation gets enough visitors.

In A/B tests, the impact of the design as a whole is tracked, not individual elements – even though many design elements might be changed on variations simultaneously.

TIP: Keep in mind that testing is all about comparing the performances of variations. It’s recommended not to make too many changes between the control and variation versions of the page at the same time. You should limit the number of changes to better understand the impact of the results. In the long term, a continuous improvement process will lead to better and lasting performance.

What is multivariate testing?

magic-cube-378543_1280

Multivariate tests or multi-variant tests are the same as A/B tests in their core mechanism and philosophy. The difference is that multivariate testing allows you to compare a higher number of variables and the interactions between each other. In other words, you can test and track changes to multiple sections on a single page.

For multivariate testing, you need to identify a few key page sections and then create variations for those sections specifically. You aren’t creating variations of a whole page as you do while A/B testing.

TIP: Use multivariate testing when several element combinations on your website or landing page are called into question.

Multivariate testing reveals more information about how these changes to multiple sections interact with one another. In multivariate tests, website traffic is split into each possible combination of a page – where the effectiveness of the changes is measured.

It’s very common to use multivariate testing to optimize an existing website or landing page without making a significant investment in redesign.

Although this type of testing can be perceived as an easier way of experimentation – keep in mind that multivariate testing is more complicated than traditional A/B testing.

Multivariate tests are best suited for more advanced testers because they give many more possibilities of combinations for visitors to experience on your website. Too many changes on a page at once can quickly add up. You don’t want to be left with a very large number of combinations that must be tested.

Multivariate test example

Let’s say that you’ve decided to run a multivariate test on one of your landing pages. You choose to change two elements on your landing page. On the first variation, you swap an image for a video, and on the second variation, you swap the image for a slider.

For each page variation, you add another version of the headline. This means that now you have three versions of the main content and two versions of the headline. This is equal to six different combinations of the landing page.

	Image	Video	Slider
Headline 1	Combination 1	Combination 2	Combination 3
Headline 2	Combination 4	Combination 5	Combination 6

After only changing two sections, you quickly have six variations. This is where multivariate testing can get tricky.

When to use multivariate testing?

Multivariate tests are recommended for sites with a large amount of daily traffic. You will need a site with a high volume of traffic to test multiple combinations, and it will take a longer time to obtain meaningful data from the test.

mvt — AB Tasty’s reporting allows you to weigh up each element’s impact on the conversion rate

The multivariate testing method will allow you to incrementally improve an existing design, while the test results can be used to apply to a larger website or landing page redesign.

What is multipage testing?

Multipage testing is an experimentation method similar to standard A/B testing. As we’ve discussed, in A/B testing, changes can be made to one specific page or to a group of pages.

If the changed element appears on several pages, you can choose whether or not to change it on each page. However, if the element is on several pages but it’s not identical, appears at a different place or has a different name, you’ll have to set up a multipage test.

Multipage tests allow you to implement changes consistently over several pages.

This means that multipage tests allow you to link together variations of different pages and are especially useful when funnel testing.

In multipage tests, site visitors are funneled into one funnel version or the other. You need to track the way visitors interact with the different pages they are shown so you can determine which funnel variation is the most effective.

You must ensure that the users see a consistent variation of changes throughout a set of pages. This is key to getting usable data and allows one variation to be fairly tested against another.

Multipage test example

Let’s say you want to conduct a multipage test with a free shipping coupon displayed in the funnel at different places. You want to run the results of this test against the original purchase funnel without a coupon.

For example, you could offer visitors a free shipping coupon on a product category page – where they can see “Free shipping over €50” as a static banner on the page. Once the visitor adds a product to the shopping cart, you can show them a new dynamic message according to the cart balance – “Add €X to your cart for free shipping”.

In this case, you can experiment with the location of the message (near the “Proceed to checkout” button, near the “Continue shopping” button, near the shipping cost for his order or somewhere else) and with the call-to-action variations of the message.

This kind of test will help you understand visitors’ purchase behavior better – i.e. how does the placement of a free shipping coupon reduce shopping cart abandonment and increase sales? After enough visitors come to the end of the purchase funnel through the different designs, you will be able to compare the effect of design styles easily and effectively.

How to test successfully?

Remember that the pages being tested need to receive substantial traffic so the tests will give you some relevant data to analyze.

Whether you use A/B testing, split testing, multivariate testing or multipage testing to increase your conversion rate or performance, remember to use them wisely.

Each type of test has its own requirements and is uniquely suited to specific situations, with advantages and disadvantages.

Using the proper test for the right situation will help you get the most out of your site and the best return on investment for your testing campaign. Even though testing follows a scientific method, there is no need for a degree in statistics when working with AB Tasty.

You might also like...

See all

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Article

2min read

New Visual Studio Code Extension: Dev-Friendly Experimentation & Personalization

Julie Dumont

Feb 21, 2025

Subscribe to
our Newsletter

Article

Jun 7, 2023

8min read

10 Generative AI Ideas for Your Experimentation Roadmap

Emily Healy

Artificial intelligence has been a recurring theme for decades. However, it’s no longer science fiction – it’s a reality.

Since OpenAI launched its own form of generative AI, ChatGPT, in November 2022, the world has yet to stop talking about its striking capabilities. It’s particularly fascinating to see just how easy it is to get results after interacting with this bot which is comprised of deep-learning algorithms for natural language processing.

Even Google quickly followed by launching a new and experimental project, Gemini, to revolutionize its own Search. By harnessing the power of generative AI and the capacity of large language models, Google is seeking to take its search process to the next level.

Given the rapid growth of this technological advancement over the past few months, it’s time that we talk about generative AI in the context of A/B testing and experimentation.

Whether you’re curious about how AI can impact your experiments or are ready for inspiration we’ll discuss some of our ideas around using AI for A/B testing, personalization, and conversion rate optimization.

What is generative AI?

Generative AI is a type of artificial intelligence that doesn’t have programming limitations, which allows it to generate new content (think ChatGPT). Instead of following a specific, pre-existing dataset, generative AI learns from indexing extensive data, focusing on patterns and using deep learning techniques and neural networks to create human-like content based on its learnings.

The way algorithms capture ideas is similar to how humans gather inspiration from previous experiences to create something unique. Based on the large amounts of data used to craft generative AI’s learning abilities, it’s capable of outputting high-quality responses that are similar to what a human would create.

However, some concerns need to be addressed:

Biased information: Artificial intelligence is only as good as the datasets used to train it. Therefore if the data used to train it has biases, it may create “ideas” that are equally biased or flawed.
Spreading misinformation: There are many concerns about the ethics of generative AI and sharing information directly from it. It’s best practice to fact-check any content written by AI to avoid putting out false or misleading information.
Content ownership: Since content generated with AI is not generated by a human, can you ethically can claim it as your own idea? In a similar sense, the same idea could potentially be generated elsewhere by using a similar prompt. Copywriting and ownership are then called into question here.
Data and privacy: Data privacy is always a top-of-mind concern. With the new capabilities of artificial intelligence, data handling becomes even more challenging. It’s always best practice to avoid using sensitive information with any form of generative AI.

By keeping these limitations in mind, generative AI has the potential to streamline processes and revolutionize the way we work – just as technology has always done in the past.

10 generative AI uses for A/B testing

In the A/B testing world, we are very interested in how one can harness these technological breakthroughs for experimentation. We are brainstorming a few approaches to re-imagine the process of revolutionizing digital customer experiences to ultimately save time and resources.

Just like everyone else, we started to wonder how generative AI could impact the world of experimentation and our customers. Here are some ideas, some of them concrete and some more abstract, as to how artificial intelligence could help our industry:

DISCLAIMER: Before uploading information into any AI platform, ensure that you understand their privacy and security practices. While AI models strive to maintain a privacy standard, there’s always the risk of data breaches. Always protect your confidential information.

1. Homepage optimization

Your homepage is likely the first thing your visitors will see so optimization is key to staying ahead of your competitors. If you want a quick comparison of content on your homepage versus your competitors, you can feed this information into generative AI to give it a basis for understanding. Once your AI is loaded with information about your competitors, you can ask for a list of best practices to employ to make new tests for your own website.

2. Analyze experimentation results

Reporting and analyzing are crucial to progressing on your experimentation roadmap, but it’s also time-consuming. By collecting a summary of testing logs, generative AI can help highlight important findings, summarize your results, and potentially even suggest future steps. Ideally, you can feed your A/B test hypothesis as well as the results to show your thought process and organization. After it recognizes this specific thought process and desired results, it could aid in generating new test hypotheses or suggestions.

3. Recommend optimization barriers

Generative AI can help you prioritize your efforts and identify the most impactful barriers to your conversion rate. Uploading your nonsensitive website performance data gathered from your analytics platforms can give AI the insight it needs into your performance. Whether it suggests that you update your title tags or compress images on your homepage, AI can quickly spot where you have the biggest drop-offs to suggest areas for optimization.

4. Client reviews

User feedback is your own treasure trove of information for optimization. One of the great benefits of AI that we already see is that it can understand large amounts of data quickly and summarize it. By uploading client reviews, surveys and other consumer feedback into the database, generative AI can assist you in creating detailed summaries of your users’ pain points, preferences and levels of satisfaction. The more detailed your reviews – the better the analysis will be.

5. Chatbots

Chatbots are a popular way to communicate with website visitors. As generative AI is a large language model, it can quickly generate conversational scripts, prompts and responses to reduce your brainstorming time. You can also use AI to filter and analyze conversations that your chatbot is already having to determine if there are gaps in the conversation or ways to enhance its interaction with customers.

6. Translation

Language barriers can limit a brand that has a presence in multiple regions. Whether you need translations for your chatbot conversations, CTAs or longer form copy, generative AI can provide you with translations in real time to save you time and make your content accessible to all zones touched by your brand.

7. Google Adwords

Speed up brainstorming sessions by using generative AI to experiment with different copy variations. Based on the prompts you provide, AI can provide you with a series of ideas for targeting keywords and creating copy with a particular tone of voice to use with Google Adwords. Caution: be sure to double-check all keywords proposed to verify their intent.

8. Personalization

Personalized content can be scaled at speed by leveraging artificial intelligence to produce variations of the same messages. By customizing your copy, recommendations, product suggestions and other messages based on past user interactions and consumer demographics, you can significantly boost your digital consumer engagement.

9. Product Descriptions

Finding the best wording to describe why your product is worth purchasing may be a challenge. With generative AI, you can get more ambitious with your product descriptions by testing out different variations of copy to see which version is the most promising for your visitors.

10. Predict User Behavior

Based on historical data from your user behavior, generative AI can predict behavior that can help you to anticipate your next A/B test. Tailoring your tests according to patterns and trends in user interaction can help you conduct better experiments. It’s important to note that predictions will be limited to patterns interpreted by past customer data collected and uploaded. Using generative AI is better when it’s used as a tool to guide you in your decision-making process rather than to be the deciding force alone.

The extensive use of artificial intelligence is a new and fast-evolving subject in the tech world. If you want to leverage it in the future, you need to start familiarizing yourself with its capabilities.

Keep in mind that it’s important to verify the facts and information AI generates just as you carefully verify data before you upload. Using generative AI in conjunction with your internal experts and team resources can assist in improving ideation and efficiency. However, the quality of the output from generative AI is only as good as what you put in.

Is generative AI a source of competitive advantage in A/B testing?

The great news is that this technology is accessible to everyone – from big industry leaders like Google to start-ups with a limited budget. However, the not-so-great news is that this is available to everyone. In other words, generative AI is not necessarily a source of competitive advantage.

Technology existing by itself does not create more value for a business. Rather, it’s the people driving the technology who are creating value by leveraging it in combination with their own industry-specific knowledge, past experiences, data collection and interpretation capabilities and understanding of customer needs and pain points.

While we aren’t here to say that generative AI is a replacement for human-generated ideas, this technology can definitely be used to complement and amplify your already-existing skills.

Leveraging generative AI in A/B testing

From education to copywriting or coding – all industries are starting to see the impact that these new software developments will have. Leveraging “large language models” is becoming increasingly popular as these algorithms can generate ideas, summarize long forms of text, provide insights and even translate in real-time.

Proper experimentation and A/B testing are at the core of engaging your audience, however, these practices can take a lot of time and resources to accomplish successfully. If generative AI can offer you ways to save time and streamline your processes, it might be time to use it as your not-so-secret weapon. In today’s competitive digital environment, continually enhancing your online presence should be at the top of your mind.

Want to start optimizing your website? AB Tasty is the best-in-class experience optimization platform that empowers you to create a richer digital experience – fast. From experimentation to personalization, this solution can help you activate and engage your audience to boost your conversions.

You might also like...

See all

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Article

2min read

New Visual Studio Code Extension: Dev-Friendly Experimentation & Personalization

Julie Dumont

Feb 21, 2025

Subscribe to
our Newsletter

Article

Jun 5, 2023

10min read

How Long Should You Run an A/B Test?

Anthony Brebion

One of the most popular questions when starting with experimentation is: How long should an A/B test run before you can draw conclusions from it?

Determining the ideal A/B test duration can be a challenge for most businesses. You have to factor in your business cycles, traffic flow, the sample size needed and be aware of other business campaigns.

Even if you reach your sample size in a few days… is it okay to end your test then? How long should you really wait?

In this article, we will discuss potential mishaps if your testing cycle is too short, give insights into which factors you need to consider and share advice on finding the best duration for your A/B tests.

Looking for fast statistical reliability? At AB Tasty, we provide a free A/B test duration calculator, which also has capabilities for a sample size calculator.

What happens if you end an A/B test too soon?

The underlying question is a crucial one and can be summed up as follows: At what point can you end a test that appears to be yielding results?

The answer depends on the relevance of the analysis and on the actual benefits of the test.

In fact, it’s not all that unusual to see tests yield good results during the trial phase and no longer see those results once the modifications are introduced.

In most cases, a disappointing observation of this nature comes down to an error during the trial phase: the test was ended too soon and the results at that point were misleading.

Let’s look at an example that illustrates the nature of the problem.

How long to run an A/B test

The graph above shows the change in the conversion rate of two versions of a page that were the subject of a test. The first version appears to break away and perform extremely well. The discrepancy between the two versions is gradually eroded as time goes by – two weeks after the starting point there’s hardly any observable difference at all.

This phenomenon where the results converge is a typical situation: the modification made does not have a real impact on conversion.

There is a simple explanation for the apparent outperformance at the start of the test: it’s unusual for the samples to be representative of your audience when the test starts. You need time for your samples to incorporate all internet user profiles, and therefore, all of their behaviors.

If you end the test too soon and allow your premature data to be the deciding factor, your results will quickly show discrepancies.

How to determine the duration of your A/B test

Now that the problem has been aired let’s have a look at how you can avoid falling into this trap.

The average recommended A/B testing time is 2 weeks, but you should always identify key factors relevant to your own conversion goals to determine the best length for a test that will meet your goals.

Let’s discuss several criteria you should use as a foundation to determine when you can trust the results you see in your A/B testing:

The statistical confidence level
The size of the sample
The representativeness of your sample
The test period and the device being tested

1. The statistical confidence level

All A/B testing solutions show a statistical reliability indicator that measures the probability of the difference in the results observed between each sample not being a matter of chance.

This indicator, which is calculated using the Chi-squared test, is the first indicator that should be used as a basis. It is used by statisticians to assert that a test is deemed reliable when the rate is 95% or higher. So, it is acceptable to make a mistake in 5% of cases and for the results of the two versions to be identical.

Yet, it would be a mistake to use this indicator alone as a basis for assessing the appropriate time to end a test.

For the purposes of devising the conditions necessary to assess the reliability of a test, this is not sufficient. In other words, if you have not reached this threshold then you cannot make the decision. Additionally, once this threshold has been reached, you still need to take certain precautions.

It’s also important to understand what the Chi-squared test actually is: a way of rejecting or not rejecting what is referred to as the null hypothesis.

This, when applied to A/B testing, is when you say that two versions produce identical results (therefore, there’s no difference between them).

If the conclusion of the test leads you to reject the null hypothesis then it means that there is a difference between the results.

However, the test is in no way an indication of the extent of this difference.

2. The size of the sample

There are lots of online tools that you can use to calculate the value of Chi-squared by giving, as the input parameters, the four elements necessary for its calculation (within the confines of a test with two versions).

AB Tasty can provide you with our own sample size calculator for you to find the value of Chi-squared.

By using this tool, we have taken an extreme example in order to illustrate this exact problem.

Sample size required for A/B testing

In this diagram, the Chi-squared calculation suggests that sample 2 converts better than sample 1 with a 95% confidence level. Having said that, the input values are very low indeed and there is no guarantee that if 1,000 people were tested, rather than 100, you would still have the same 1 to 3 ratio between the conversion rates.

It’s like flipping a coin. If there is a 50% probability that the coin will land heads-up or tails-up, then it’s possible to get a 70 / 30 distribution by flipping it just 10 times. It’s only when you flip the coin a very large number of times that you get close to the expected ratio of 50 / 50.

So, in order to have faith in the Chi-squared test, you are advised to use a significant sample size.

You can calculate the size of this sample before beginning the test to get an indication of the point at which it would be appropriate to look at the statistical reliability indicator. There are several tools online that you could use to calculate this sample size.

In practice, this can turn out to be difficult, as one of the parameters to be given is the % improvement expected (which is not easy to evaluate). But, it can be a good exercise to assess the pertinence of the modifications being envisaged.

Pro Tip: The lower the expected improvement rate, the greater the sample size needed to be able to detect a real difference.

If your modifications have a very low impact, then a lot of visitors will need to be tested. This serves as an argument in favor of introducing radical or disruptive modifications that would probably have a greater impact on the conversion.

img_548ab40f4fb20

3. The representativeness of your sample

If you have a lot of traffic, then getting a sufficiently large sample size is not a problem and you will be able to get a statistical reliability rate in just a few days, sometimes just two or three.

Having said that, ending a test as soon as the sample size and statistical reliability conditions have been met is no guarantee that results in a real-life situation are being reproduced.

The key point is to test for as long as you need to in order for all of your audience segments to be included.

Actually, the statistical tests operate on the premise that your samples are distributed in an identical fashion. In other words, the conversion probability is the same for all internet users.

But this is not the case: the probability varies in accordance with different factors such as the weather, the geographical location and also user preferences.

There are two very important factors that must be taken into account here: your business cycles and traffic sources.

Your business cycles

Internet users do not make a purchase as soon as they come across your site. They learn more, they compare, and their thoughts take shape. One, two or even three weeks might elapse between the time they are the subject of one of your tests and the point at which they convert.

If your purchasing cycle is three weeks long and you have only run the test for one week, then your sample will not be representative. As the tool records visits from all internet users, they may not record the conversions of those that are impacted by your test.

Therefore, you’re advised to test over at least one business cycle and ideally two.

Your traffic sources

Your sample must incorporate all of your traffic sources including emails, sponsored links and social networks. You need to make sure that no single source is over-represented in your sample.

Let’s take a concrete situation: if the email channel is a weak source of traffic but significant in terms of revenue and you carry out a test during an email campaign, then you are going to include internet users who have a stronger tendency to make a purchase in your sample.

This would no longer be a representative sample. It’s also crucial to know about major acquisition projects and, if possible, not to test during these periods.

The same goes for tests during sales or other significant promotional periods that attract atypical internet users. You will often see less marked differences in the results if you re-do the tests outside these periods.

It turns out that it’s quite difficult to make sure that your sample is representative, as you have little control over the kind of internet users who take part in your test.

Thankfully, there are two ways of overcoming this problem.

The first is to extend the duration of your test more than is necessary in order to get closer to the normal spread of your internet users.
The second is to target your tests so that you only include a specific population group in your sample. For example, you could exclude all internet users who have come to you as a result of your email campaigns from your samples, if you know that this will distort your results. You could also target only new visitors so that you do not include visitors who have reached an advanced stage in their purchasing process (AKA visitors who are likely to convert regardless of which variation they see).

4. Other elements to keep in mind

There are other elements to bear in mind in order to be confident that your trial conditions are as close as they can be to a real-life situation: timing and the device.

Conversion rates can vary massively on different days of the week and even at different times of the day. Therefore, you’re advised to run the test over complete periods.

In other words, if you launch the test on a Monday morning then it should be stopped on a Sunday evening so that a normal range of conversions is respected.

In the same way, conversion rates can vary enormously between mobiles, tablets and desktop computers. So with devices, you’re advised to test your sites or pages specifically for each device. This is easy to accomplish by using the targeting features to include or exclude the devices if your users show very different browsing and purchasing behavior patterns.

These elements should be taken into account so that you do not end your tests too soon and get led astray by a faulty analysis of the results.

They also explain why certain A/A tests carried out over a period of time that is too short, or during a period of unusual activity, can present differences in results and also differences in statistical reliability, even when you may not have made any modifications at all.

The ideal A/B test duration

Running and A/B testing requires a thorough consideration of various factors such as your personal conversion goals, statistical significance, sample size, seasonality, campaigns, traffic sources, etc. All factors deserve equal attention when determining the best practices for your business.

Just remember to be patient, even if you reach your sample size early. You may be surprised by the ending results.

As A/B testing is an iterative process, continuous experimentation and conversion rate optimization will lead to better results over time.

You might also like...

See all

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Article

2min read

New Visual Studio Code Extension: Dev-Friendly Experimentation & Personalization

Julie Dumont

Feb 21, 2025

Subscribe to
our Newsletter

Article

Jun 1, 2023

4min read

5 Mistakes to Avoid When Selecting an EOP

AB Tasty

If you’re an e-commerce company, you know better than anyone how important it is to optimize your website and have the best possible user experience.

You’ve heard about experience optimization platforms (EOPs) and how they can improve your website’s performance, enhance customer loyalty and increase order size. We know you’re excited – and so are we!

But, before you dive headfirst into the dizzying world of EOP selection, let’s go through the most common mistakes e-commerce companies make so you can avoid them!

Mistake #1: Focusing on quantity over quality

You might be tempted to select an EOP that offers a ton of features, capabilities, and integrations. After all, more is better… right? Not necessarily.

When it comes to experience optimization, quality > quantity.

It’s especially important to make sure that what the EOP provides is right for YOUR company and its goals.

It’s easy to be seduced by an EOP that boasts 50 integrations or hundreds of features. But, a platform that overwhelms you with options and doesn’t deliver real value will end up costing you precious time and resources.

Opt for the platform that offers the most effective features that help you achieve your specific goals.

Mistake #2: Not considering customer success

You’ve worked hard to hire the smartest people for the job and have full faith in your employees. They can probably figure out any problem that comes up. But, when it comes to experience optimization, things don’t always go as planned.

This is why it’s so important to choose a platform that offers excellent customer support.

Look for a platform that not only provides responsive support but also has a community of users with people you genuinely enjoy working with. A platform like this can offer strategic guidance in alignment with your goals.

Mistake #3: Overlooking scalability

We know that part of the reason why you’re looking for an EOP is to grow your customer base. One of the best ways you can do that is by selecting an EOP that can grow with you.

Not every EOP is designed to scale, so make sure you’re choosing one that accommodates your company’s growth goals and can accompany you along the way.

Look for a platform that offers scalability within its plans and can support increased traffic, as well as new features.

Mistake #4: Ignoring mobile and app optimization

More and more customers are choosing to shop from their phones, which means mobile optimization for all devices is critical.

A poorly optimized mobile experience can lead to lost sales – and we know you can’t afford that!

Choose a platform that prioritizes mobile optimization and delivers a seamless experience across all devices, whether it’s web or app.

Omnichannel is the way of the future. A poor mobile experience can leave a bad taste in a customer’s mouth.

Mistake #5: Neglecting data and analytics

Robust data and analytics help you make better business decisions and are absolutely essential for effective experience optimization.

Go for a platform that relies on solid statistical models, creates reports based on your needs as an e-commerce company, and lets you choose the KPIs that are most relevant to your business.

A platform that delivers valuable insights will enable you to make smarter, data-driven decisions, which can lead to higher ROI and revenue.

Selecting the right EOP for e-commerce

Selecting the right EOP is more crucial than ever for e-commerce companies.

We know you have a lot of options out there, big goals to achieve, and even bigger dreams of where you want to take your company next. AB Tasty is here to help guide you through the confusing, often complicated process of selecting an EOP that fits with your business now and in the future.

By choosing a company that addresses the five areas above, you can rest assured that you’re on the right track.

Looking for a solution that addresses all five of these areas? AB Tasty is the best-in-class experience optimization platform that empowers you to create a richer digital experience – fast. From experimentation to personalization, this solution can help you activate and engage your audience to boost your conversions.

You might also like...

See all

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Article

2min read

New Visual Studio Code Extension: Dev-Friendly Experimentation & Personalization

Julie Dumont

Feb 21, 2025

Subscribe to
our Newsletter

Article

May 30, 2023

8min read

Digital Consumer Engagement: Activate Your Mid-Market Audience

Emily Healy

One of the biggest challenges for mid-market businesses is keeping your website visitors engaged. At this point, you’ve grown your brand enough to have a significant amount of traffic, but are in search of the right strategy to nudge your passive visitors into paying customers.

When you have a significant number of passive visitors, enticing them to take action is crucial to growing your commercial activity, achieving your business objectives and creating loyalty.

After working diligently to get traffic on your website, how do you engage these visitors?

In this article, we will answer that burning question with some practical strategies to ultimately improve your conversions.

What are passive users?

Passive users are visitors on your website that do not take any action.

These individuals may browse through some content on your site, but they leave before impacting your conversion rates in a positive way.

They are simply just browsing your web pages instead of filling out forms, subscribing to newsletters, engaging with CTAs, or making a purchase.

How to identify passive users

To engage passive users, you have to identify them.

Website analytics tools, such as Google Analytics, can help you identify these particular users by recognizing patterns when you’re analyzing your users’ behavior.

With analytics tools, you can see:

Which pages are more likely to attract passive visitors
The number of passive visitors on your website
Which pages attract active users
The actions taken on each page
The amount of time visitors spend on each page
Where your visitors come from
And more

Note: Many analytics tools provide more metrics than you can manage. In other words, it’s easy to get lost in the sea of data and metrics. It’s important to identify key performance indicators to measure your user behavior consistently.

This information can help you understand user behavior and recognize what motivates visitors to take action if you analyze it correctly. Once you can identify these motivators, you can begin to create a strategy to employ similar tactics on other pages for consistency in your engagement.

Identifying and understanding your digital consumers’ behavior is the first step to reaching your conversion goals.

Why you need to engage your audience

Converting passive visitors into paying customers is crucial for success. A higher conversion rate means more revenue and growth opportunities.

To recap: in order to engage your audience, you will need to understand them based on the data you collect from their behavior. This collection of information will help you develop best practices and develop guidelines for your future endeavors. Not to mention, you will also see what doesn’t work.

Mastering this first step to engaging your audience will, at the same time, help you to create a more seamless and attractive digital customer experience.

Whether your goal is to convert more leads on your landing page, increase your sign-up form conversions, increase brand awareness or nudge visitors to add items to their shopping cart, you must capture your audience to make these objectives a reality.

As your company increases conversions, your brand reputation will grow in parallel. A great reputation attracts new customers and creates further brand loyalty.

Now the question is, how do you engage your users in the mid-market?

How to engage visitors with widgets

Unfortunately, maximizing your digital consumer engagement won’t happen overnight. Based on your consumer research on your website, you will need to create a plan to optimize your digital customer journey.

Offering incentives such as discounts, free trials or exclusive offers is one great way to engage passive website visitors. By doing this you are providing a reason for users to take action.

How can you achieve such engagement at a mid-market level? Leveraging widgets.

With little to no-code needed, widgets are the best tools you can use for your website optimization strategy. Deploying flexible, visually appealing, and impactful components, such as widgets, is a foolproof way to increase digital consumer engagement.

Here are a few ideas and advice on how you can implement widgets to boost your engagement:

Alert and notify with a pop-in

An exit-intent pop-in is one popular widget to set up on your page that can have a real impact on retaining your visitors before they leave. By displaying important information that your visitor might have missed, you are giving them another opportunity to benefit from your message.

Le Slip Français, a French clothing manufacturer, displayed a seasonal exit pop-in with free delivery during Valentine’s Day, leading to a 22% increase in clicks.

Leverage geotargeting banners

Having multiple regions requires you to differentiate your communication. With geo-targeted banners, you can maximize your chances of a conversion by sending the right message to the right digital consumer in the right location.

This type of widget can be used in a variety of ways such as creating a customizable banner to offer discounts or services at the closest physical store to the user, which could entice them to go to your location.

Be aware of overuse

It’s important to think of the right time and place for your information. Put yourself in the shoes of your digital audience and ask yourself what information would be enticing and what would be overwhelming.

Creating space with your messages is essential to avoid burdening visitors with too many pop-ins and banners. Even if you’re offering discounts for various reasons, be sure not to go overboard.

Test what works and what doesn’t

Every audience is unique. What works for one company may not work for the next.

This is why A/B testing your new ideas is important to find out what your audience likes. Without testing, a bias based on your personal likes and dislikes can get in the way of data-driven decision-making. Experimentation is the only way to find what works for your customers without making prior assumptions.

Want to start enticing your customers with widgets and leveraging A/B testing on your website? AB Tasty is the best-in-class experience optimization platform that empowers you to create a richer digital experience – fast. From experimentation to adding widgets, this solution can help you activate and engage your audience to boost your conversions.

Benefits of engaging digital consumers with widgets

Providing your visitors with additional opportunities to engage with your website can bring a number of benefits to your business, which include:

Drive conversions

Your visitors have already communicated their interest in your products by visiting your webpage. Now they need to be convinced that your products or service will be worth it to nudge them down the funnel.

Find out what influences your digital consumers’ decision-making with different tactics:

A countdown timer creates excitement for digital consumers. Alerting visitors that there is a timely discount helps create a sense of urgency that will help them make a decision, fast.
Adding social proof to your product pages is a popular scarcity and urgency tactic used by many marketers to help draw attention to certain products. By letting visitors know that their peers are also interested in the same product, they will feel a sense of comfort knowing they aren’t alone.

As an example, NYX cosmetics leveraged AB Tasty’s social proof banner widget on their product pages to create desire. Without any coding knowledge, NYX created this test and saw a 2x increase in their transaction rate compared to the original version.

Increase basket size

Another benefit of engaging users on your website is increasing the average basket size.

By adding widgets that propose complimentary products to your consumer’s purchases, you are actively encouraging them to add more to their basket while highlighting benefits that they may receive.

Let non-subscribers know they are eligible for a discount if they upgrade or give visibility to a free shipping discount by adding a progress bar.

Decathlon, a French sporting goods retailer, implemented a progress bar using AB Tasty’s easy-to-use widgets. By offering this special “gift” of free shipping to customers who filled the progress bar, they saw a 10% increase in the transaction rate.

Build loyalty

Building loyalty is an ever-growing challenge with the growing market saturation. Post-purchase, you should focus your attention on building customer loyalty and creating new brand ambassadors.

One strategy to increase loyalty is by implementing widgets to collect information about their experience in the form of a pop-up survey. An exit pop-in is a great way to capture your customers’ attention while the transaction process is still fresh in their minds.

Unicef, an agency of the United Nations responsible for providing humanitarian and developmental aid to children worldwide, used an iframe widget to collect information about their digital consumer’s experience quickly and saw over 300 answers in a short period of time.

unicef survey

Engaging your digital audience

When you have fewer interactive elements, there’s nothing in place to entice visitors to stay on your website. This means ultimately ending up with more passive customers.

Meanwhile, placing easy-to-use widgets is a great way to grab your customers’ attention and give them the nudge they need to engage more with your website. Getting your digital consumers to make the switch from passive to active will eventually make a positive impact on the KPIs that matter to you most.

If you’re looking to drive conversions, increase your basket size or drive loyalty – widgets may be the perfect addition to your optimization roadmap.

You might also like...

See all

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Article

2min read

New Visual Studio Code Extension: Dev-Friendly Experimentation & Personalization

Julie Dumont

Feb 21, 2025

Subscribe to
our Newsletter

Article

May 25, 2023

10min read

When to Make the Leap from Client- to Server-Side Testing

Rowan Haddad

As companies start out their experimentation journey, they may find themselves experimenting with small changes on their website such as the design of a CTA and other small changes to explore how they perform and their impact on key KPIs such as conversion rate and transaction.

However, as an experimentation program matures, companies are more likely to want to delve deeper into more sophisticated types of testing which require more expertise and a more advanced tool.

This is the point when many companies are ready to take the plunge from client- to server-side experimentation.

In this article, we will explore when is the right time to make the shift to server-side testing and discuss the importance of running these types of tests by outlining the various scenarios where server-side experiments are more appropriate.

Client- vs server-side testing

Before we go deeper into server-side testing, we will quickly point out the differences between client- and server-side tests to understand why you need both types of testing as your experimentation and optimization program evolves.

In client-side testing is where experimentation occurs on the client side through Javascript that runs in the browser. Therefore, client-side tools enable you to create variations of your pages by changing the content sent by your server to users in the web browser. The user then gets one variation of your altered content based on your targeting rules.

Put simply, all the work happens at the level of the browser thanks to Javascript. Because of this, client-side testing is usually best for surface-level changes such as layout, design and colors to measure their performance and impact on key KPIs.

Meanwhile, all the work in server-side testing happens at the server level rather than the browser. In other words, it’s your server that randomly sends a user the modified variation. As a result, the experimentation tool works on the server instead of inside your users’ browsers.

Perhaps one of the biggest benefits of client-side tests is that it’s easy to implement and no special expertise is required to run these tests on the front-end of the website.

However, because of its advanced capabilities, server-side tests require technical expertise and coding skills and so developers are usually the ones running these tests on the back-end.

What are the benefits of server-side testing?

Before we go into when you should be using server-side tests, we will outline some of their benefits in order to better understand their use cases.

Omnichannel experimentation

Client-side solutions are usually limited to devices with web browsers, whether that’s on desktop, mobile or tablet. This means you cannot run experiments on mobile apps or connected devices.

Server-side solutions, for their part, allow you to experiment across multiple channels including mobile apps which significantly widens your playing field and opens up numerous opportunities to A/B test beyond web browsers so you can cover all your bases.

Enhanced performance

Perhaps one of the biggest advantages of server-side tests is the absence of the flicker effect, which is often a major downside associated with client-side solutions.

The flicker effect occurs when the original page loads and is replaced by the test variation and is usually seen by the user.

Unlike client-side tools that require adding a Javascript tag to your pages, you don’t need to add this tag in server-side tools. That’s because experiments are rendered on the server before being pushed to the client and so all the major work occurs at the server level- it’s not noticeable on the client side.

In other words, during a server-side test, the variation is retrieved from the server and delivered to the user’s browser. As a result, no modifications take place on the front-end or browser, so there’s no flickering effect

Endless testing opportunities

Your teams have a lot more options to work with when it comes to server-side testing as it enables you to modify all aspects of your site.

As server-side tests are rendered from the back-end server, you can test more complex dynamic content unlike with client-side tests where it’s not easy to test such content and may damage the user experience if done on the client-side.

In other words, you can build tests that are much more complex that go deeper within your tech stack beyond the scope of UI or cosmetic changes to unlock a whole new world of experimentation.

With such deep experimentation capabilities, teams can thoroughly test all facets of a product to validate its functionality such as evaluating its underlying features, algorithms and back-end logic.

The next section will go over these various use cases where you should consider running server-side testing instead.

When does it make sense to move to server-side testing?

As we’ve mentioned, server-side tests are usually used to run more advanced types of tests and experiment deeply within your tech stack to thoroughly explore how a product works.

Put simply, server-side experimentation solutions allow teams to conduct more robust and secure experiments that are focused on modifying a product’s functions.

Here are some use cases where server side testing is recommended over client-side testing:

Run experiments on your mobile app

As already mentioned, one of the key advantages of server-side testing is enabling omnichannel and cross-platform experimentation.

As client-side solutions rely on Javascript and cookies, it’s not possible to use them to test on native mobile apps and you’re limited to devices that have a default web browser.

This means to run experiments on your mobile app, you will need a more advanced server-side testing solution to handle mobile app technologies, which are more complex and vastly different from web technologies.

Moreover, because server-side testing works on both web applications and mobile apps, you can run the same test serving the same variations irrespective of which channel they’re using. This allows you to compare data from each channel and optimize the user experience across all the different touch points accordingly.

Finally, if you use feature flags to conduct server-side testing on mobile apps, you can bypass the tedious and time-consuming app store approval. Put simply, feature flags enable you to turn functionality on or off remotely without redeploying code to app stores and waiting for its approval or having to wait for all changes to be ready at the same time to release your own changes.

Test your search algorithms

Server-side A/B tests are efficient for testing deeper level modifications related to the backend and the architecture of your website.

This is the case with search algorithms which require modification to your existing code. E-commerce websites usually turn to server-side testing to ensure that customers can easily browse through their website and find the product they’re looking for.

Thus, search algorithms are key in getting customers to find the product they’re looking for by providing a seamless search experience that eventually gets them to purchase.

For example, you can test what is displayed to customers in their search results, whether this should be based on pricing, popularity or reviews or whether you need to prioritize products based on what customers purchased/liked/viewed in the past.

Server-side testing allows you to create such complex scenarios and rules to provide customers with more personalized reocmmendations and optimize their search experience on your site. These are more difficult to test through client-side solutions as search pages are based on the search query and so they are rendered dynamically.

Thus, server-side testing offers more comprehensive testing and allows you to experiment with multiple algorithms by modifying the existing code.

Optimize your product recommendations

Similarly, with server-side tools, you can test multiple product recommendation algorithms typically found at the bottom of product pages to measure which types of layouts generate the most sales or increase average order value. For example, is it better to promote similar products, the most popular products or those that were recently viewed?

Such recommendations are based on a variety of criteria, like browsing history and on your own PIM (Product Information Management) system and other data sources. Server-side testing will allow you to test these multiple algorithms to uncover the best performing selection of recommended products.

Test data-sensitive pages

Server-side testing is great for testing on pages where data security is vital as it ensures that the data remains safe internally within the server without worrying about a security threat.

This is ideal to test on e-commerce payment pages or for banking applications, for example, or any other web pages or apps that contain sensitive data.

You can also use feature flags to test out a new payment method on a subset of users and see how they adapt before rolling it out to everyone else.

Determine the ideal form length

This is especially important for SaaS businesses that rely on free trial and request demo forms to gather information from their visitors. Thus, they need to determine the best length for such forms without risking drop-offs but still being able to gather all the necessary information about a prospect.

Server-side testing is the way to go in this scenario as your forms are directly linked to how your database is structured. If a field is obligatory, you will not be able to hide it using JavaScript because the form’s validation will fail server-side.

As a result, testing the length and complexity of a form that has the highest positive impact on conversion rates should be done on the server-side.

This also applies to other websites that utilize such forms from visitors such as hotel reservation or insurance sites. Note that different sectors will require different and more or less information depending on the type and purpose of the information being gathered.

Test the limit for free shipping

E-commerce businesses should be able to determine the order value over which free shipping is possible. This is important as shipping costs are one of the major causes for cart abandonment.

Therefore, since shipping cost is one of the determining factors in a customer’s purchase decision, companies should test out various cart value thresholds to find out the optimal limit for free shipping to improve transaction rates.

Since shipping costs are usually rendered dynamically from the back-end server, you will need to test them on the server-side. Any modifications made should have an impact on all of the following steps, and should be managed server-side.

Validate your features

Server-side tests allow you to conduct feature testing to validate your product features by rolling out variations of your feature to different groups of users to evaluate its performance before going for a general release.

With the help of feature flags, you can run server-side A/B tests and segment your users by directing them to one variation or the other. If anything goes wrong with a variation, you can easily roll it back by disabling the flag before it further impacts the user experience.

Finally, according to the feedback generated from these users, you can then optimize your features and release them to all your users with the confidence that it matches customers requirements.

Server-side tests and feature flags

The best way to run server-side tests is through feature flags. By decoupling deployment from release, you can conduct server-side A/B tests by rolling out new features to a small group of users. You can then measure their performance on this group of users before rolling out to everyone else.

While server-side testing requires technical expertise and coding skills, it’s not only relevant to tech teams. Often, non-technical staff team up with product teams to define experiments, which are then executed by the engineers. Once implemented, an experiment can usually be controlled, monitored, and analyzed via a dashboard.

With the right feature management solution, all teams across an organization can run server-side tests with the help of an easy-to-use dashboard without facing any technical hassles.

Client- vs server-side testing: Context matters

In the end, it’s important to note that it’s not a question of whether server-side is better than client-side. They’re both complementary approaches and whichever one a company chooses depends on which is better suited for its optimization and larger business goals.

In other words, one type of testing doesn’t replace the other. It’s a matter of looking at the type of experiment you want to run and which type is more suited in that particular context and which teams are looking to run the experiment- marketers tend to favor and use client-side testing more often while product managers and developers usually opt for server-side testing for their experimentation needs. It also depends on the resources businesses have at hand and the maturity of their optimization programs.

To ensure your website is optimized and provides a seamless user experience, having both testing techniques at hand is key to surviving in a competitive digital world.

Both types of testing are indispensable to help you build great products that will bring in maximum revenue. The key is to use both together to heighten productivity and achieve maximum impact.

You might also like...

See all

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Article

2min read

New Visual Studio Code Extension: Dev-Friendly Experimentation & Personalization

Julie Dumont

Feb 21, 2025

Subscribe to
our Newsletter

Article

May 24, 2023

3min read

AB Tasty Teams Up with Google Cloud

John Hughes

We are thrilled to announce that AB Tasty is now an official Google Cloud Partner, the next step in the strengthening of our partnership with Google.

With this partnership, our clients and partners can now benefit from the combination of AB Tasty’s powerful digital experience tools and Google’s secure and reliable platform.

Whether you are looking to engage and activate your clients or create resilient digital products that deliver exceptional customer experiences and strong business results, AB Tasty’s suite of products is now easier than ever to access.

From experimentation to personalization, server-side testing, feature flags, search and product recommendations, our solutions are designed to help you achieve your goals and take your digital experiences further.

Google Cloud Marketplace

With AB Tasty available on the Google Cloud Marketplace (GCM), product and marketing teams can easily integrate AB Tasty’s digital experience tools into their existing Google Cloud vendor relationships, without the need for time-consuming vendor certification processes.

The streamlined marketplace transactions and standardized terms also provide added peace of mind, requiring less involvement from legal and contracting teams, and enabling product teams to deploy AB Tasty solutions quickly and efficiently.

It also means businesses can be more flexible and agile if they want access to our experimentation tools to quickly set up accounts and contracts through Google. Businesses with an annual commitment to Google Cloud can easily incorporate AB Tasty solutions into their existing packages and quotes, while also gaining access to established vendor relationships with Google Cloud.

Additionally, the partnership allows businesses to take advantage of Google Cloud’s flexible billing options, as GCM automatically breaks up annual commitments into monthly charges on their Google Cloud Platform (GCP) bill. This makes it easier for businesses to manage their digital marketing budgets and adjust them according to their needs.

Finally, it enables businesses to count their AB Tasty departmental spend against their broader GCP Cloud infrastructure committed spend, helping them maximize their overall investment in Google Cloud. Partnering with Google Cloud and AB Tasty provides businesses with a more streamlined and cost-effective approach to digital marketing.

Our Strengthened Relationship with Google

Following the announcement of our integration with Google Analytics 4 (GA4) and AB Tasty being chosen at one of Google’s vendors of choice after the sunset of Google Optimize, this news takes our relationship with Google to new heights:

“As the first platform to integrate bi-directional synchronization with Google Analytics 4, and being one of the three partners highlighted by Google for the Optimize sunset, AB Tasty has established strong ties with the tech giant.” said Remi Aubert, CEO and Co-founder of AB Tasty. “Moreover, as one of the top Google Cloud partners and the largest digital native on the platform, AB Tasty is the ideal partner for navigating the Google Optimize sunset.”

As AB Tasty continues to help clients improve each touchpoint of their customer experience journey, we’re pleased to cement our ties with Google so strongly. Our work with Google fuels the growth of our software for experimentation, personalization, recommendation and intelligent search. In an increasingly competitive market, this strategic alliance provides businesses with the competitive edge they need.

You might also like...

See all

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Article

2min read

New Visual Studio Code Extension: Dev-Friendly Experimentation & Personalization

Julie Dumont

Feb 21, 2025

Subscribe to
our Newsletter

Article

May 22, 2023

7min read

A/B Test Hypothesis Definition, Tips and Best Practices

Emily Healy

Incomplete, irrelevant or poorly formulated A/B test hypotheses are at the root of many neutral or negative tests.

Often we imagine that doing A/B tests to improve your e-commerce site’s performance means quickly changing the color of the “add to cart” button will lead to a drastic increase in your conversion rate, for example. However, A/B testing is not always so simple.

Unfortunately, implementing random changes to your pages won’t always significantly improve your results – there should be a reason behind your web experiments.

This brings us to the following question: how do you know which elements to experiment with and how can you create an effective AB test hypothesis?

Determine the problem and the hypothesis

Far too few people question the true origins of the success (or failure) of the changes they put in place to improve their conversion rate.

However, it’s important to know how to determine both the problem and the hypothesis that will allow you to obtain the best results.

Instead of searching for a quick “DIY” solution, it’s often more valuable in the long term to take a step back and do two things:

Identify the real problem – What is the source of your poor performance? Is it a high bounce rate on your order confirmation page, too many single-page sessions, a low-performing checkout CTA or something more complex?
Establish a hypothesis – This could show the root of the problem. For example, a great hypothesis for A/B testing could be: “Our customers do not immediately understand the characteristics of our products when they read the pages on our e-commerce site. Making the information more visible will increase the clicks on the “add-to-cart” button.”

The second step may seem very difficult because it requires a capacity for introspection and a critical look at the existing site. Nevertheless, it’s crucial for anyone who wants to see their KPIs improve drastically.

If you’re feeling a bit uncomfortable with this type of uncertainty around creating an effective hypothesis, know that you’ve come to the right place.

What is an A/B test hypothesis?

Technically speaking, the word hypothesis has a very simple definition:

“A proposal that seeks to provide a plausible explanation of a set of facts and which must be controlled against experience or verified in its consequences.”

The first interesting point to notice in this definition is “the set of facts to be explained.” In A/B testing, a hypothesis must always start with a clearly identified problem.

A/B tests should not be done randomly, or you risk wasting time.

Let’s talk about how to identify the problem:

Web analytics data – While this data does not explain digital consumers’ behavior exactly, it can highlight conversion problems (identifying abandoned carts, for example) and help prioritize the pages in need of testing.
Heuristic evaluation and ergonomic audit – These analyses allow you to assess the site’s user experience at a lower cost using an analysis grid.
User tests – This qualitative data is limited by the sample size but can be very rich in information that would not have been detected with quantitative methods. They often reveal problems understanding the site’s ergonomics. Even if the experience can be painful given the potential for negative remarks, it will allow you to gather qualified data with precise insights.
Eye tracking or heatmaps – These methods provide visibility into how people interact with items within a page – not between pages.
Customer feedback – As well as analyzing feedback, you can implement tools such as customer surveys or live chats to collect more information.

The tactics above will help you highlight the real problems that impact your site’s performance and save you time and money in the long run.

A/B test hypothesis formula

Initially, making an A/B test hypothesis may seem too simple. At the start, you mainly focus on one change and the effect it produces. You should always respect the following format: If I change this, it will cause that effect. For example:

Changing (the element being tested) from ___________ to ___________ will increase/decrease (the defined measurement).

At this stage, this formula is only a theoretical assumption that will need to be proven or disproven, but it will guide you in solving the problem.

An important point, however, is that the impact of the change you want to bring must always be measurable in quantifiable terms (conversion rate, bounce rate, abandonment rate, etc.).

Here are two examples of hypotheses phrased according to the formula explained above and that can apply to e-commerce:

Changing our CTA from “BUY YOUR TICKETS NOW” to “TICKETS ARE SELLING FAST – ONLY 50 LEFT!” will improve our sales on our e-commerce site.
Shortening the sign-up form by deleting optional fields such as phone and mailing address will increase the number of contacts collected.

In addition, when you think about the solution you want to implement, include the psychology of the prospect by asking yourself the following:

What psychological impact could the problem cause in the digital consumer’s mind?

For example, if your problem is a lack of clarity in the registration process which impacts the purchases, then the psychological impact could be that your prospect is confused when reading information.

With this in mind, you can begin to think concretely about the solution to correct this feeling on the client side. In this case, we can imagine that one fix could be including a progress bar that shows the different stages of registration.

Be aware: the psychological aspect should not be included when formulating your test hypothesis.

Once you have gotten the results, you should then be able to say whether it is true or false. Therefore, we can only rely on concrete and tangible assumptions.

Best practice for e-commerce optimization based on A/B hypotheses

There are many testable elements on your website. Looking into these elements and their metrics can help you create an effective test hypothesis.

We are going to give you some concrete examples of common areas to test to inspire you on your optimization journey:

HOMEPAGE

The header/main banner explaining the products/services that your site offers can increase customers’ curiosity and extend their time on the site.
A visible call-to-action appearing upon arrival will increase the chance visitors will click.
A very visible “about” section will build prospects’ trust in the brand when they arrive on the site.

PRODUCT SECTIONS

Filters save customers a lot of time by quickly showing them what they are looking for.
Highlighting a selection of the most popular products at the top of the sections is an excellent starting point for generating sales.
A “find out more” button or link under each product will encourage users to investigate.

PRODUCT PAGES

Product recommendations create a more personal experience for the user and help increase their average shopping cart
A visible “add to cart” button will catch the prospect’s attention and increase the click rate.
An “add to cart and pay” button saves the customer time, as many customers have an average of one transaction at a time.
Adding social sharing buttons is an effective way of turning the product listing into viral content.

Want to start A/B testing elements on your website? AB Tasty is the best-in-class experience optimization platform to help you convert more customers by leveraging intelligent search and recommendations to create a richer digital experience – fast. From experimentation to personalization, this solution can help you achieve the perfect digital experience with ease.

CART PAGE

The presence of logos such as “Visa certified” enhances customer confidence in the site.
A very visible button/link to “proceed to payment” greatly encourages users to click.

PAYMENT

A single page for payment reduces the exit rate.
Paying for an order without registration is very much appreciated by new prospects, who are not necessarily inclined to share their personal information when first visiting the site.
Having visibility over the entire payment process reassures consumers and will nudge them to finalize their purchase.

These best practices allow you to build your A/B test hypotheses by comparing your current site with the suggestions above and see what directly impacts conversion performance.

The goal of creating an A/B test hypothesis

The end goal of creating an A/B test hypothesis is to identify quickly what will help guarantee you the best results. Whether you have a “winning” hypothesis or not, it will still serve as a learning experience.

While defining your hypotheses can seem complex and methodical, it’s one of the most important ways for you to understand your pages’ performance and analyze the potential benefits of change.

You might also like...

See all

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Article

2min read

New Visual Studio Code Extension: Dev-Friendly Experimentation & Personalization

Julie Dumont

Feb 21, 2025

Subscribe to
our Newsletter

Article

May 16, 2023

6min read

How to Leverage Disruption in Experimentation | Ben Labay

Rowan Haddad

Ben Labay outlines essential frameworks for a more strategic, tactical and disruptive approach to experimentation

With two degrees, in Evolutionary Behavior and Conservation Research Science, Ben Labay spent a decade in academia with a wide-ranging background in research and experimentation dealing with technical data work.

Now as CEO of experimentation and conversion optimization agency Speero, Ben describes his work in experimentation as his “geek-out” area which is customer experience research and dealing with customer data.

At Speero, Ben works to scope and run research and test program strategies for companies including Procter & Gamble, ADP, Codecademy, MongoDB, Toast and many others around the world.

AB Tasty’s VP Marketing Marylin Montoya spoke with Ben on how to create mechanisms for companies to not only optimize but also be more disruptive when it comes to web experimentation to drive growth.

Here are some of the key takeaways from their conversation.

Consider a portfolio way of management in experimentation

Inspired by Jim Collins’ and Jerry I. Porras’ book “Built to Last”, Ben discusses a framework that the book provides on the ways a company can grow based on the best practices from 18 successful companies.

He identifies one big pillar that many organizations are often neglecting: experimentation. To tackle this, Ben suggests taking a portfolio management way of doing experimentation made up of three portfolio tags which provide a solution spectrum around iterative changes for optimization.

The first level consists of making small tweaks or changes to a website based on customer feedback such as improving layouts and the second which includes more substantial types of changes such as new content pieces.

But there’s a bigger third level which Ben refers to as more “disruptive” and “innovative” such as a brand new product or pricing model that can serve as a massive learning experience.

With three different levels of change, it’s important to set a clear distribution of time spent on each level and have alignment among your team.

In the words of Ben, “Let’s put 20% of our calories over into iterating, 20% onto substantial and 20/30/ or 40% over on disruptive. And that map – that framework has been really healthy to use as a tool to get teams on the same page.”

For Ben, applying such a framework is key to getting all teams on the same page as it helps ensure companies are not under-resourcing disruptive and “big needle movers”. Velocity of work is important, he argues, but so is quality of ideas.

Let your goal tree map guide you

Every A/B test or personalization campaign needs to be fed with good ingredients which determine the quality of the hypothesis.

“Every agency, every in-house company researches. We do research. We collect data, we have information, we get insights and then they test on insights. But you can’t stop there.” Ben says.

The trick is not to stop at the insights part but to actually derive a theme based on those insights. This will allow companies to pick underlying strengths and weaknesses to map them into their OKRs.

For example, you may have a number of insights like a page underperforming, users are confused about pricing and social proof gets skipped over. The key is to conduct a thematic analysis and look for patterns based on these different insights.

Consequently, it’s important for companies to create a goal tree map to help them understand how things cascade down and to become more tactical and SMART about their goals and set their OKRs accordingly to organize and make sense of the vast amount of data.

When the time comes to set up a testing program, teams will have a strategic testing roadmap for a particular theme that links to these OKRs. This helps transform the metrics into more actionable frameworks.

And at the end of each quarter, companies can evaluate their performance based on this scorecard of metrics and how the tests they ran during the quarter impacted these metrics.

Build engagement and efficiency into your testing program strategy

The main value prop of testing centers around making profit but Ben advocates for a second value prop which revolves around how a business operates. This requires shifting focus to efficiency and how different teams across an organization work together.

Ben parallels the A/B testing industry with Devops as it strives to bring in elements from the DevOps cultural movement when we refer to a culture of experimentation and being data-driven. In many ways, this echoes the DevOps methodology, which is focused on breaking down silos between development and operation teams to enhance collaboration and efficiency between these teams. “The whole idea is to optimize the efficiency of a big team working together”, Ben says.

This means organizations should take a hard look at their testing program and the components that make up the program which includes getting the right people behind it. It’s also about becoming more customer-centric and embracing failure.

Ben refers to this as the “programmatic side” of the program which serves as the framework or blueprint for decision making. It helps to answer questions like “how do I organize my team structure?” or “what is my meeting cadence with the team?”

Ultimately, it’s about changing and challenging your current process and transforming your culture internally by engaging your team within testing your program and the way you’re using data to make decisions.

What else can you learn from our conversation with Ben Labay?

Ways to get out of a testing rut
How to structure experimentation meetings to tackle roadblocks
How experimentation relates to game theory
The importance of adopting a actionable framework for decision making

About Ben Labay

Ben Labay combines years of academic and statistics training with customer experience and UX knowledge. Currently, Ben is the CEO at Speero. With two degrees in Evolutionary Behavior and Conservation Research Science (resource management), Ben started his career in academia, working as a staff researcher at the University of Texas focused on research and data modeling. This helped form the foundation for his current passion and work at Speero, which focuses on helping organizations make decisions using customer data.

About 1,000 Experiments Club

The 1,000 Experiments Club is an AB Tasty-produced podcast hosted by Marylin Montoya, VP of Marketing at AB Tasty. Join Marylin and the Marketing team as they sit down with the most knowledgeable experts in the world of experimentation to uncover their insights on what it takes to build and run successful experimentation programs.

You might also like...

See all

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Article

2min read

New Visual Studio Code Extension: Dev-Friendly Experimentation & Personalization

Julie Dumont

Feb 21, 2025

Subscribe to
our Newsletter

Article

May 11, 2023

17min read

AB Tasty’s JavaScript Tag Performance and Report Analysis

AB Tasty

Hello! I am Léo, Product Manager at AB Tasty. I’m in charge, among several things, of our JavaScript tag that is currently running on thousands of websites for our clients. As you can guess, my roadmap is full of topics around data collection, privacy and… performance.

In today’s article, we are going to talk about JavaScript tag performance, open-data monitoring and competition. Let’s go!

Performance investigation

As performance has become a big and hot topic during the past few years, mainly thanks to Google’s initiative to deploy their Core Web Vitals, my team and I have been focused a lot on that. We’ve changed a lot of things, improved many parts of our tag and reached excellent milestones. Many of our users have testified of their satisfaction around that. I have already made a (long) series of blog articles about that here. Sorry though, it’s only in French. 😊🥖

From time to time, we get tickled by competitors about a specific report around performance that seems to show us as underperforming based on some metrics. Some competitors claim that they are up to 4 times faster than us! And that’s true, I mean, that’s what the report shows.

You can easily imagine how devastating this can be for the image of my company and how hard it could be for our sales team when a client draws this card. This is especially demoralizing for me and my team after all the work we’ve pushed through this topic during the last few years.

Though it was the first feeling I got when seeing this report, I know for a fact that our performance is excellent. We’ve reached tremendous improvements after the release of several projects and optimizations. Today all the benchmarks and audits I run over our customers’ websites show very good performance and a small impact on the famous Core Web Vitals.

Also, it’s very rare that a customer complains about our performance. It can happen, that’s for sure, but most of the time all their doubts disappear after a quick chat, some explanations and hints about optimization best practices.

But that report is still there, right? So maybe I’m missing something. Maybe I’m not looking at the correct metric. Maybe I’ve only audited customers where everything is good, but there’s a huge army of customers that don’t complain that our tag is drastically slowing their website down.

One easy way to tackle that would be to say that we are doing more with our tag than our competitors do.

Is CRO the same as analytics?

On the report (I promise I will talk about it in depth below 😄), we are grouped in the Analytics Category. However, Conversion Rate Optimization isn’t the same as Analytics. An analytics tool only collects data while we activate campaigns, run personalizations, implement widgets, add pop-ins and more. In this sense, our impact will be higher.

Let’s talk about our competitors: Even though we have the best solution out there (😇), our competitors do more or less the same things as us by using the same technics with the same limits and issues. Therefore, it’s legit to compare us with the same metrics. It might be true that we do a bit more than they do, but in the end, this shouldn’t explain a 4x difference in our performance.

Back then, and before digging into the details, I took the results of the report with humility. Therefore, my ambition was to crawl the data, analyze websites where their tag is running and try to find what they do better than us. We call that retro-engineering, and I find it healthy as it would help to have a faster website for everyone.

My engagement with my management was to find where we had a performance leak and solve it to be able to decrease our average execution time and get closer to our competitors.

But first, I needed to analyze the data. And, wow, I wasn’t prepared for that.

The report

The report is a dataset that is being monthly generated by The HTTP Archive. Here is a quote from their About Page:

“Successful societies and institutions recognize the need to record their history – this provides a way to review the past, find explanations for current behavior, and spot emerging trends. In 1996, Brewster Kahle realized the cultural significance of the Internet and the need to record its history. As a result he founded the Internet Archive which collects and permanently stores the Web’s digitized content.”

“In addition to the content of web pages, it’s important to record how this digitized content is constructed and served. The HTTP Archive provides this record. It is a permanent repository of web performance information such as size of pages, failed requests, and technologies utilized. This performance information allows us to see trends in how the Web is built and provides a common data set from which to conduct web performance research.”

Every month, they run a Lighthouse audit on millions of websites and generate a dataset containing the raw results.

As it is open-source and legit, it can be used by anyone to draw data visualization and ease access to this type of data.

That’s what the inventor of Google Lighthouse, Patrick Hulce, has done. Through his website, GitHub, he provides a nice visualization of this huge dataset and allows anyone to dig into details through several categories such as Analytics, Ads, Social Media and more. As I said, you’ll find the CRO tools in the Analytics category.

The website is fully open-source. The methodology is known and can be accessed.

So, what’s wrong with the report?

Well, there’s nothing technically wrong with it. We could find it disappointing that the dataset isn’t automatically updated every month, but the repository is open-source, so anyone motivated could do it.

However, this is only displaying the data in a fancy manner and not providing any insights or deep analysis of it. Any flaw or inconsistency will remain hidden and it could lead to a situation where a third party is seen as having bad performance compared to others when it is not necessarily the case.

One issue though, not related to the report itself, is the flaw an average could bring with it. That’s also something we are all aware of but that we tend to forget. If you take 10 people, 9 of them earn 800€ a month but one is earning 12 million euros a month, then we could conclude that everyone earns 1.2 million euros per month. Statistically right, but sounds a bit wrong, doesn’t it? More on that in a minute.

Knowing that, it was time to get my hands a bit dirty. With my team, we downloaded the full dataset from February 2023 to run our own audit and understand where we had performance leaks.

Note that downloading the full dataset is something we have been doing regularly for about one and a half years to monitor our trend. However, this time I decided to dig into the February 2023 report in particular.

The analysis

On this dataset, we could find the full list of websites running AB Tasty that have been crawled and the impact our tag had on them. To be more accurate, we have the exact measured execution time of our tag, in milliseconds.

This is what we extracted. The pixellated column is the website URL. The last column is the execution time in milliseconds.

With the raw data, we were able to calculate a lot of useful metrics.

Keep in mind that I am not a mathematician or anything close to a statistics expert. My methodology might sound odd, but it’s adequate for this analysis.

Average execution time

This is the first metric I get — the raw average for all the websites. That’s probably very close, if not equal, to what is used by the thirdpartyweb.today website. We already saw the downside of having an average, however, it’s still an interesting value to monitor.

Mean higher half and mean lower half

Then, I split the dataset in half. If I have 2000 rows, I create two groups of 1000 rows. The “higher” one and the “lower” one. It helps me have a view of the websites where we perform – the worst compared to the best. Then, I calculate the average of each half.

The difference between the two halves

The difference between the two halves is important as it shows the disparity within the dataset. The closer it is, the less extreme values we have.

The number of websites with a value above 6k ms

It’s just an internal metric we follow to give us a mid-term goal of having 0 websites above this value.

The evolution of the last dataset

I compute the evolution between the last dataset I have and the current. It helps me see if we get better in general, as well as how many websites are leaving or entering the chart.

The results

These are the results that we have:

Here are their corresponding graphs:

This is the evolution between October 2022 and February 2023:

Watch out: Logarithmic scale! Sorted by February 2023 execution time from left to right.

The figures say it all. But, if I can give a global conclusion, it’s that we made tremendous improvements in the first six months and staled a bit after with finer adjustments (the famous 80/20 of Pareto’s).

However, after the initial fall, two key figures are important.

First of all, the difference between the two halves is getting very close. This means that we don’t have a lot of potential performance leaks anymore (features that lead to an abnormal increase in the execution time). This is our first recent win.

Then, the evolution shows that in general, and except for the worst cases, it is steady or going down. Another recent win.

Digging into the details

What I have just shared is the raw results without having a look at the details of each row and each website that is being crawled.

However, as we say, the devil is in the details. Let’s dig in a bit.

Let’s focus on the websites where AB Tasty takes more than six seconds to execute.

Six seconds might sound like a lot (and it is), but don’t forget that the audit simulates a low-end CPU which is not representative of the average device. Instead, it shows the worst-case scenario.

In the February 2023 report, there are 33 of them. This is an average execution time of 19877 ms. I quickly identified that:

27 of them are from the same AB Tasty customer
One of them is abtasty.com and the total execution of resources coming from *abtasty.com on this website is very high 😊
Two others are also coming from one singular AB Tasty customer

In the end, we have only 5 customers on this list (but still 33 websites, don’t get me wrong).

Let’s now try to group up these two customers with duplicates to see the impact on the average. The customer with 27 duplicates also has websites that are below the 6k ms mark, but I’m going to ignore it for now (and to ease things up).

For each of the two customers with duplicates, I’m going to compute the average of all their duplicates. For the first one, the result is 21671 ms. For the second, the result is 14708 ms.

I’m also going to remove abtasty.com, which is not relevant.

With the new list, I went from 1223 ms for the full list average to 1005 ms. I just improved our average by more than 200 ms! 🎉

Wait, what? But you’re just removing the worst websites. Obviously, you are getting better…

Yep, that’s true. That’s cheating for sure! But, the point of this whole article is to demonstrate that data doesn’t say it all.

Let’s talk first about what is happening with this customer that has 27 duplicates.

The same tag has been deployed on more than 50 very different websites! You might not be very familiar with AB Tasty, so let me explain why this is an issue.

You might have several websites which have the same layout (that’s often the case when you have different languages). It makes sense to have the same tag on these different domains to be able to deploy the same personalizations on all of them at once. That’s not the most optimal way of doing it, but as of today, that’s the easiest way to do it with our tool.

However, if your websites are all different, there is absolutely no point in doing that. You are going to create a lot of campaigns (in this case, hundreds!) that will almost never be executed on the website (because it’s not the correct domain) but are still at least partially included in the tag. So our tag is going to spend its time checking hundreds of campaigns that have no chance to execute as the URL is rarely going to be valid.

Though we are working on a way to block this behavior (as we have alternatives and better options), it will take months before it disappears from the report.

Note: If you start using AB Tasty, you will not be advised to do that. Furthermore, the performance of your tag will be far better than that.

Again, I didn’t take the time to group all the duplicated domains as it is pointless, the goal was to demonstrate that it is easy to show better performance if we exclude anomalies that are not representative. We can imagine that we would improve more than 200+ ms by keeping only one domain.

I took the most obvious case, but a quick look at the rest of the dataset showed me some other examples.

The competitors’ figures

Knowing these facts and how our score might look worse than it is because of one single anomaly, I started looking into our competitors’ figures to see if they have the same type of issue.

I’m going to say it again: I’m not trying to say that we are better (or worse) than any of our competitors here, that’s not my point. I’m just trying to show you why statistics should be deeply analyzed to avoid any interpretation mistakes.

Let’s start by comparing AB Tasty’s figures for February 2023 with the same metrics for one of them.

Competitor's figures

In general, they look a bit better, right? Better average and even the means for each half is better (and the lower half by a lot!).

However, between the two halves, the factor is huge: 24! Does it mean that depending on your usage, the impact of their tag might get multiplied by 24?

If I wanted to tease them a little bit, I would say that when testing the tag on your website, you might find excellent performance but when starting to use it intensely you might face serious performance drops.

But, that would be interpreting a very small part of what the data said.

Also, they have more than twice the number of websites that are above the 6k ms mark (again: this mark is an AB Tasty internal thing). And that is by keeping the duplicates in AB Tasty’s dataset that we discussed just before! They also have duplicates, but not as many as we do.

A first (and premature) conclusion is that they have more websites with a big impact on performance but at the same time, their impact is lower in general.

Now that I know that in our case we have several customers that have duplicates, I wanted to check if our competitors have the same. And this one does – big time.

Among the 2,537 websites that have been crawled, 40% of them belong to the same customer. This represents 1,016 subdomains of the same domain.

How does this impact their score?

Well, their customer wasn’t using the solution at the moment the data was collected (I made sure of it by visiting some of the subdomains). This means that the tag wasn’t doing anything at all. It was there, but inactive.

The average execution time of these 1,016 rows in the dataset is 59 ms!! 😭 It also has a max value of 527 ms and a min value of 25 ms.

I don’t need to explain why this “anomaly” interestingly pulls down their average, right?

The 1,016 subdomains are not fake websites at all. I’m not implying that this competitor cheated on purpose to look better- I’m sure they didn’t. It is just a very nice coincidence for them, whether they are aware of it or not.

To finish, let’s compare the average of our two datasets after removing these 1,016 subdomains.

AB Tasty is at 1223 ms (untouched list) when this competitor is now at… 1471 ms.

They went from 361 ms better to 248 ms worse. I told you that I can let the figures say whatever I want. 🙂

I would have a lot of other things to say about these datasets, but I didn’t run all the analysis that could have been done here. I already spent too much time on it, to be honest.

Hopefully, though, I’ve made my point of showing that the same dataset can be interpreted in a lot of different manners.

What can we conclude from all of this?

The first thing I want to say is: TEST IT.

Our solution is very easy to implement. You simply put the tag on your website and run an audit. To compare, you can put another tool’s tag on your website and run the same audit. Run it several times with the same conditions and compare. Is the second tool better on your website? Fine, then it will probably perform better for your specific case.

Does a random report on the web says that one solution is better than another? Alright, that’s one insight, but you should either crunch the data to challenge it or avoid paying too much attention to it. Just accepting the numbers as it is displayed (or worse: advertised…) might make you miss a big part of the story.

Does AB Tasty have a bad performance?

No, it doesn’t. Most of our customers never complained about performance and some are very grateful for the latest improvements we’ve released on this topic.

So, some customers are complaining?

Yes. This is because sometimes AB Tasty can have a lower performance depending on your usage. But, we provide tools to help you optimize everything directly from our platform. We call this the Performance Center. It is a full section inside the platform and is dedicated to showing you which campaign is impacting your performance and what you can do to improve it. Just follow the guidelines and you’ll be good. It’s a very innovative and unique feature in the market, and we are very proud of it.

Though, I must admit that a few customers (only a few) have unrealistic expectations about performance. AB Tasty is a JS tag that is doing DOM manipulations, asynchronous checks, data collection and a lot of fancy stuff. Of course, it will impact your website more than a simple analytics tool will. The goal for you is to make sure that the effect of optimizing your conversions is higher than what it costs you in terms of performance. And it will be the same, whatever the CRO tool you are using, except if you use a server-side tool like Flagship by AB Tasty, for example.

I am convinced that we should aim towards a faster web. I am very concerned about my impact on the environment, and I’m trying to keep my devices as long as possible. My smartphone is 7 years old (and I’m currently switching to another one that is 10 years old) and my laptop isn’t very recent either. So, I know that a slow website can be a pain.

Final Remarks

Let me assure you that at AB Tasty we are fully committed to improving our performance because our customers are expecting us to do it, because I am personally motivated to do it, and because that is a very fun and interesting challenge for the team (and also because my management asks me to do it 😅 ).

Also, kudos to HTTP Archive which does very important work in gathering all this data and especially sharing it with everyone. Kudos to Patrick Hulce who took the time to build a very interesting website that helps people have a visual representation of HTTP Archive’s data. Kudos to anyone that works to build a better, faster and more secure web, often for free and because that’s what they believe in.

Want to test our tool for yourself? AB Tasty is the complete platform for experimentation, content personalization, and AI-powered recommendations equipped with the tools you need to create a richer digital experience for your customers — fast. With embedded AI and automation, this platform can help you achieve omnichannel personalization and revolutionize your brand and product experiences.