Tag: EN

Article

May 11, 2023

17min read

AB Tasty’s JavaScript Tag Performance and Report Analysis

AB Tasty

Hello! I am Léo, Product Manager at AB Tasty. I’m in charge, among several things, of our JavaScript tag that is currently running on thousands of websites for our clients. As you can guess, my roadmap is full of topics around data collection, privacy and… performance.

In today’s article, we are going to talk about JavaScript tag performance, open-data monitoring and competition. Let’s go!

Performance investigation

As performance has become a big and hot topic during the past few years, mainly thanks to Google’s initiative to deploy their Core Web Vitals, my team and I have been focused a lot on that. We’ve changed a lot of things, improved many parts of our tag and reached excellent milestones. Many of our users have testified of their satisfaction around that. I have already made a (long) series of blog articles about that here. Sorry though, it’s only in French. 😊🥖

From time to time, we get tickled by competitors about a specific report around performance that seems to show us as underperforming based on some metrics. Some competitors claim that they are up to 4 times faster than us! And that’s true, I mean, that’s what the report shows.

You can easily imagine how devastating this can be for the image of my company and how hard it could be for our sales team when a client draws this card. This is especially demoralizing for me and my team after all the work we’ve pushed through this topic during the last few years.

Though it was the first feeling I got when seeing this report, I know for a fact that our performance is excellent. We’ve reached tremendous improvements after the release of several projects and optimizations. Today all the benchmarks and audits I run over our customers’ websites show very good performance and a small impact on the famous Core Web Vitals.

Also, it’s very rare that a customer complains about our performance. It can happen, that’s for sure, but most of the time all their doubts disappear after a quick chat, some explanations and hints about optimization best practices.

But that report is still there, right? So maybe I’m missing something. Maybe I’m not looking at the correct metric. Maybe I’ve only audited customers where everything is good, but there’s a huge army of customers that don’t complain that our tag is drastically slowing their website down.

One easy way to tackle that would be to say that we are doing more with our tag than our competitors do.

Is CRO the same as analytics?

On the report (I promise I will talk about it in depth below 😄), we are grouped in the Analytics Category. However, Conversion Rate Optimization isn’t the same as Analytics. An analytics tool only collects data while we activate campaigns, run personalizations, implement widgets, add pop-ins and more. In this sense, our impact will be higher.

Let’s talk about our competitors: Even though we have the best solution out there (😇), our competitors do more or less the same things as us by using the same technics with the same limits and issues. Therefore, it’s legit to compare us with the same metrics. It might be true that we do a bit more than they do, but in the end, this shouldn’t explain a 4x difference in our performance.

Back then, and before digging into the details, I took the results of the report with humility. Therefore, my ambition was to crawl the data, analyze websites where their tag is running and try to find what they do better than us. We call that retro-engineering, and I find it healthy as it would help to have a faster website for everyone.

My engagement with my management was to find where we had a performance leak and solve it to be able to decrease our average execution time and get closer to our competitors.

But first, I needed to analyze the data. And, wow, I wasn’t prepared for that.

The report

The report is a dataset that is being monthly generated by The HTTP Archive. Here is a quote from their About Page:

“Successful societies and institutions recognize the need to record their history – this provides a way to review the past, find explanations for current behavior, and spot emerging trends. In 1996, Brewster Kahle realized the cultural significance of the Internet and the need to record its history. As a result he founded the Internet Archive which collects and permanently stores the Web’s digitized content.”

“In addition to the content of web pages, it’s important to record how this digitized content is constructed and served. The HTTP Archive provides this record. It is a permanent repository of web performance information such as size of pages, failed requests, and technologies utilized. This performance information allows us to see trends in how the Web is built and provides a common data set from which to conduct web performance research.”

Every month, they run a Lighthouse audit on millions of websites and generate a dataset containing the raw results.

As it is open-source and legit, it can be used by anyone to draw data visualization and ease access to this type of data.

That’s what the inventor of Google Lighthouse, Patrick Hulce, has done. Through his website, GitHub, he provides a nice visualization of this huge dataset and allows anyone to dig into details through several categories such as Analytics, Ads, Social Media and more. As I said, you’ll find the CRO tools in the Analytics category.

The website is fully open-source. The methodology is known and can be accessed.

So, what’s wrong with the report?

Well, there’s nothing technically wrong with it. We could find it disappointing that the dataset isn’t automatically updated every month, but the repository is open-source, so anyone motivated could do it.

However, this is only displaying the data in a fancy manner and not providing any insights or deep analysis of it. Any flaw or inconsistency will remain hidden and it could lead to a situation where a third party is seen as having bad performance compared to others when it is not necessarily the case.

One issue though, not related to the report itself, is the flaw an average could bring with it. That’s also something we are all aware of but that we tend to forget. If you take 10 people, 9 of them earn 800€ a month but one is earning 12 million euros a month, then we could conclude that everyone earns 1.2 million euros per month. Statistically right, but sounds a bit wrong, doesn’t it? More on that in a minute.

Knowing that, it was time to get my hands a bit dirty. With my team, we downloaded the full dataset from February 2023 to run our own audit and understand where we had performance leaks.

Note that downloading the full dataset is something we have been doing regularly for about one and a half years to monitor our trend. However, this time I decided to dig into the February 2023 report in particular.

The analysis

On this dataset, we could find the full list of websites running AB Tasty that have been crawled and the impact our tag had on them. To be more accurate, we have the exact measured execution time of our tag, in milliseconds.

This is what we extracted. The pixellated column is the website URL. The last column is the execution time in milliseconds.

With the raw data, we were able to calculate a lot of useful metrics.

Keep in mind that I am not a mathematician or anything close to a statistics expert. My methodology might sound odd, but it’s adequate for this analysis.

Average execution time

This is the first metric I get — the raw average for all the websites. That’s probably very close, if not equal, to what is used by the thirdpartyweb.today website. We already saw the downside of having an average, however, it’s still an interesting value to monitor.

Mean higher half and mean lower half

Then, I split the dataset in half. If I have 2000 rows, I create two groups of 1000 rows. The “higher” one and the “lower” one. It helps me have a view of the websites where we perform – the worst compared to the best. Then, I calculate the average of each half.

The difference between the two halves

The difference between the two halves is important as it shows the disparity within the dataset. The closer it is, the less extreme values we have.

The number of websites with a value above 6k ms

It’s just an internal metric we follow to give us a mid-term goal of having 0 websites above this value.

The evolution of the last dataset

I compute the evolution between the last dataset I have and the current. It helps me see if we get better in general, as well as how many websites are leaving or entering the chart.

The results

These are the results that we have:

Here are their corresponding graphs:

This is the evolution between October 2022 and February 2023:

Watch out: Logarithmic scale! Sorted by February 2023 execution time from left to right.

The figures say it all. But, if I can give a global conclusion, it’s that we made tremendous improvements in the first six months and staled a bit after with finer adjustments (the famous 80/20 of Pareto’s).

However, after the initial fall, two key figures are important.

First of all, the difference between the two halves is getting very close. This means that we don’t have a lot of potential performance leaks anymore (features that lead to an abnormal increase in the execution time). This is our first recent win.

Then, the evolution shows that in general, and except for the worst cases, it is steady or going down. Another recent win.

Digging into the details

What I have just shared is the raw results without having a look at the details of each row and each website that is being crawled.

However, as we say, the devil is in the details. Let’s dig in a bit.

Let’s focus on the websites where AB Tasty takes more than six seconds to execute.

Six seconds might sound like a lot (and it is), but don’t forget that the audit simulates a low-end CPU which is not representative of the average device. Instead, it shows the worst-case scenario.

In the February 2023 report, there are 33 of them. This is an average execution time of 19877 ms. I quickly identified that:

27 of them are from the same AB Tasty customer
One of them is abtasty.com and the total execution of resources coming from *abtasty.com on this website is very high 😊
Two others are also coming from one singular AB Tasty customer

In the end, we have only 5 customers on this list (but still 33 websites, don’t get me wrong).

Let’s now try to group up these two customers with duplicates to see the impact on the average. The customer with 27 duplicates also has websites that are below the 6k ms mark, but I’m going to ignore it for now (and to ease things up).

For each of the two customers with duplicates, I’m going to compute the average of all their duplicates. For the first one, the result is 21671 ms. For the second, the result is 14708 ms.

I’m also going to remove abtasty.com, which is not relevant.

With the new list, I went from 1223 ms for the full list average to 1005 ms. I just improved our average by more than 200 ms! 🎉

Wait, what? But you’re just removing the worst websites. Obviously, you are getting better…

Yep, that’s true. That’s cheating for sure! But, the point of this whole article is to demonstrate that data doesn’t say it all.

Let’s talk first about what is happening with this customer that has 27 duplicates.

The same tag has been deployed on more than 50 very different websites! You might not be very familiar with AB Tasty, so let me explain why this is an issue.

You might have several websites which have the same layout (that’s often the case when you have different languages). It makes sense to have the same tag on these different domains to be able to deploy the same personalizations on all of them at once. That’s not the most optimal way of doing it, but as of today, that’s the easiest way to do it with our tool.

However, if your websites are all different, there is absolutely no point in doing that. You are going to create a lot of campaigns (in this case, hundreds!) that will almost never be executed on the website (because it’s not the correct domain) but are still at least partially included in the tag. So our tag is going to spend its time checking hundreds of campaigns that have no chance to execute as the URL is rarely going to be valid.

Though we are working on a way to block this behavior (as we have alternatives and better options), it will take months before it disappears from the report.

Note: If you start using AB Tasty, you will not be advised to do that. Furthermore, the performance of your tag will be far better than that.

Again, I didn’t take the time to group all the duplicated domains as it is pointless, the goal was to demonstrate that it is easy to show better performance if we exclude anomalies that are not representative. We can imagine that we would improve more than 200+ ms by keeping only one domain.

I took the most obvious case, but a quick look at the rest of the dataset showed me some other examples.

The competitors’ figures

Knowing these facts and how our score might look worse than it is because of one single anomaly, I started looking into our competitors’ figures to see if they have the same type of issue.

I’m going to say it again: I’m not trying to say that we are better (or worse) than any of our competitors here, that’s not my point. I’m just trying to show you why statistics should be deeply analyzed to avoid any interpretation mistakes.

Let’s start by comparing AB Tasty’s figures for February 2023 with the same metrics for one of them.

Competitor's figures

In general, they look a bit better, right? Better average and even the means for each half is better (and the lower half by a lot!).

However, between the two halves, the factor is huge: 24! Does it mean that depending on your usage, the impact of their tag might get multiplied by 24?

If I wanted to tease them a little bit, I would say that when testing the tag on your website, you might find excellent performance but when starting to use it intensely you might face serious performance drops.

But, that would be interpreting a very small part of what the data said.

Also, they have more than twice the number of websites that are above the 6k ms mark (again: this mark is an AB Tasty internal thing). And that is by keeping the duplicates in AB Tasty’s dataset that we discussed just before! They also have duplicates, but not as many as we do.

A first (and premature) conclusion is that they have more websites with a big impact on performance but at the same time, their impact is lower in general.

Now that I know that in our case we have several customers that have duplicates, I wanted to check if our competitors have the same. And this one does – big time.

Among the 2,537 websites that have been crawled, 40% of them belong to the same customer. This represents 1,016 subdomains of the same domain.

How does this impact their score?

Well, their customer wasn’t using the solution at the moment the data was collected (I made sure of it by visiting some of the subdomains). This means that the tag wasn’t doing anything at all. It was there, but inactive.

The average execution time of these 1,016 rows in the dataset is 59 ms!! 😭 It also has a max value of 527 ms and a min value of 25 ms.

I don’t need to explain why this “anomaly” interestingly pulls down their average, right?

The 1,016 subdomains are not fake websites at all. I’m not implying that this competitor cheated on purpose to look better- I’m sure they didn’t. It is just a very nice coincidence for them, whether they are aware of it or not.

To finish, let’s compare the average of our two datasets after removing these 1,016 subdomains.

AB Tasty is at 1223 ms (untouched list) when this competitor is now at… 1471 ms.

They went from 361 ms better to 248 ms worse. I told you that I can let the figures say whatever I want. 🙂

I would have a lot of other things to say about these datasets, but I didn’t run all the analysis that could have been done here. I already spent too much time on it, to be honest.

Hopefully, though, I’ve made my point of showing that the same dataset can be interpreted in a lot of different manners.

What can we conclude from all of this?

The first thing I want to say is: TEST IT.

Our solution is very easy to implement. You simply put the tag on your website and run an audit. To compare, you can put another tool’s tag on your website and run the same audit. Run it several times with the same conditions and compare. Is the second tool better on your website? Fine, then it will probably perform better for your specific case.

Does a random report on the web says that one solution is better than another? Alright, that’s one insight, but you should either crunch the data to challenge it or avoid paying too much attention to it. Just accepting the numbers as it is displayed (or worse: advertised…) might make you miss a big part of the story.

Does AB Tasty have a bad performance?

No, it doesn’t. Most of our customers never complained about performance and some are very grateful for the latest improvements we’ve released on this topic.

So, some customers are complaining?

Yes. This is because sometimes AB Tasty can have a lower performance depending on your usage. But, we provide tools to help you optimize everything directly from our platform. We call this the Performance Center. It is a full section inside the platform and is dedicated to showing you which campaign is impacting your performance and what you can do to improve it. Just follow the guidelines and you’ll be good. It’s a very innovative and unique feature in the market, and we are very proud of it.

Though, I must admit that a few customers (only a few) have unrealistic expectations about performance. AB Tasty is a JS tag that is doing DOM manipulations, asynchronous checks, data collection and a lot of fancy stuff. Of course, it will impact your website more than a simple analytics tool will. The goal for you is to make sure that the effect of optimizing your conversions is higher than what it costs you in terms of performance. And it will be the same, whatever the CRO tool you are using, except if you use a server-side tool like Flagship by AB Tasty, for example.

I am convinced that we should aim towards a faster web. I am very concerned about my impact on the environment, and I’m trying to keep my devices as long as possible. My smartphone is 7 years old (and I’m currently switching to another one that is 10 years old) and my laptop isn’t very recent either. So, I know that a slow website can be a pain.

Final Remarks

Let me assure you that at AB Tasty we are fully committed to improving our performance because our customers are expecting us to do it, because I am personally motivated to do it, and because that is a very fun and interesting challenge for the team (and also because my management asks me to do it 😅 ).

Also, kudos to HTTP Archive which does very important work in gathering all this data and especially sharing it with everyone. Kudos to Patrick Hulce who took the time to build a very interesting website that helps people have a visual representation of HTTP Archive’s data. Kudos to anyone that works to build a better, faster and more secure web, often for free and because that’s what they believe in.

Want to test our tool for yourself? AB Tasty is the complete platform for experimentation, content personalization, and AI-powered recommendations equipped with the tools you need to create a richer digital experience for your customers — fast. With embedded AI and automation, this platform can help you achieve omnichannel personalization and revolutionize your brand and product experiences.

You might also like...

See all

Article

2min read

How We Made AB Tasty’s Feature Experimentation SDK OpenFeature-Compatible — And Even Easier to Adopt

Julie Dumont

Apr 30, 2025

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Subscribe to
our Newsletter

Article

May 10, 2023

11min read

Product Manager vs Product Owner: What’s the Difference?

AB Tasty

What’s the difference between a product manager and a product owner or are these roles actually one and the same?

If you’re wondering, you’re not alone. The terms are often used interchangeably and not without reason as sometimes the two roles’ responsibilities may overlap especially in small to medium-sized companies.

However, Product Manager (PM) and Product Owner (PO) are two distinct roles. Although they do share a common goal, delivering products users love, the scope of their responsibilities is actually not identical.

Generally, a product manager is responsible for the why. They shape the product roadmap based on users’ needs and desires. They are focused on business metrics and on whether the product is going in the right direction on a larger scale.

The product owner, on the other hand, is responsible for creating and managing the product backlog. They are operational and on a deadline. For them, it’s all about Scrum, back and forth with developers, and getting stuff done.

However, the lines between the two do often blur. Depending on where you work, you might be used to a variety of setups.

What are the differences in responsibilities of a product manager and a product owner?

The short answer is: it depends. Product Manager Experience

Now, I’m sure you’re thinking ‘of course! Isn’t that true for everything?’

But in this case, it really does depend on a lot of different factors. A chunk of this article will go over them in detail but first we will discuss each role to highlight the responsibilities of each one on its own to be able to then understand their differences.

The product owner role

The term “product owner” comes from Scrum- an Agile framework that helps teams structure and manage their work and solve complex problems effectively by adopting a set of values and principles. A typical Scrum team consists of Product Owner, a Scrum Master and Developers with specific accountabilities.

According to the “Scrum Guide”, the product owner is responsible for “maximizing the value of the product resulting from the work of the Scrum Team”. Consequently, this role is usually found in organizations that have adopted an Agile methodology.

Among the key responsibilities of the product owner is creating user stories for the development team to implement and ensuring that these stories meet customer requirements. In other words, the product owner advocates for customer needs and represents the voice of the customer to the development team by making sure that the right product is being built.

Some other responsibilities of the product owner include:

Manage product backlog by creating and communicating the backlog items and prioritize them accordingly to maximize value
Define and manage the product vision
Understand market and customer needs and turn them into actionable user stories
Collaborate closely with cross-functional teams such as developers to ensure that products meet customer needs and requirements. They also make sure that goals are clear and business objectives are aligned with the product vision
Act as the primary point of contact for all stakeholders and ensure they have proper buy-in on all major decisions and strategies

The product manager role

As we’ve seen, product owners are more tactical in that their focus is on maximizing value through creating and managing the product backlog. Additionally, product owners are more detail-oriented and have more of a short- to mid-term focus.

Meanwhile, product managers play a more strategic role, which means their focus is more on creating the long-term vision of a product as well as aligning the product roadmap with larger organizational goals.

They then build the product strategy in order to build a concrete product based on its overall vision that spans across the entire roadmap. Put simply, a product manager manages the entire product life cycle and oversees every aspect of its development from the early stages of user research to the moment of product launch.

The responsibilities of the product manager may vary from one organization to the next but they usually include the following tasks:

Conduct user research to determine customer needs to determine product vision
Create and align teams around the product roadmap
Decide what features to build next
Supervise teams and projects to ensure the successful launch of the product

A product manager role is usually more outward-facing in that they speak to customers to define the requirements of the product to be built while a product owner is more inward-facing as they work closely with development teams to ensure the product is being built according to these requirements.

We can look at product owners as an extension of product managers as they apply the strategies outlined by product managers and transform it into an actionable backlog.

In other words, the product manager decides what products or features to build next while the product owner oversees that developers build these products.

Do organizations need both roles?

The simple answer to this question would be it depends.

Various factors will need to be taken into account when it comes to deciding whether you need one or the other or both.

In an ideal scenario, there would be a product manager managing the product strategy and vision and a product owner responsible for executing that strategy. On the surface, it may seem that the responsibilities of the two roles may sometimes overlap as they’re working towards common goals but in reality, their day-to-day tasks differ significantly.

However, not all companies may have the sufficient resources to have two different people dedicated to each of these roles.

Indeed, many small companies may find that having two distinct roles is not necessary and the responsibilities of the product development process can be carried out solely by the product manager. However, processes may become more complex in larger companies and could require a product manager to manage the product life cycle and outline the overall strategy and a product owner overseeing the development process to ensure the tactical execution of that strategy.

It’s important to also keep in mind that the product owner role is tightly linked with the Scrum development approach. Therefore, it’s typically found in teams practicing Scrum. Product managers, for their part, exist in organizations that aren’t necessarily following a specific approach or methodology. They can operate within any framework as part of the product team while product owners in highly Agile businesses come as part of a Scrum team.

Nonetheless, many organizations struggle with the decision of whether to have one or the other or both.

The most important consideration when making this decision is focusing on outcomes and not on the titles. Organizations need to examine their objectives and any weaknesses and bottlenecks in their current processes or structures that may hinder them from achieving these objectives. Such insights are key to building your winning team.

When to consider a product manager vs product owner

Let’s not forget, there are products of all kinds: industrial products, agricultural products, services, chemical products, fashion and software products.

In the SaaS martech space, the thinking goes that all products are built the same way and that companies are all structured the same way. They tend to think there is always a Product Manager and a Product Owner behind each feature.

We’ve already mentioned some things to consider when you’re deciding whether to have both roles. In this section, we look in further detail at some of these factors that determine the right setup for your organization.

1. Momentum

The most important criteria is ‘the momentum of the company.’ What does this mean?

How new/old is the organization?

This question is an important one. It is not directly related to the size of the company or to the number of clients, either.

But rather, the right question here would be: How far or close is the company in relation to a specific “key moment”?

product owner vs product manager ab tasty

What we mean by ‘key moment’ are key phases or events in the life cycle of a company. Some examples include:

the creation of the company
its sale
new funding
tax inspection
a competitor’s release
a crisis
replacement of the CEO
… etc

All these different key stages will have an impact on how the product development is executed.

And therefore, they affect what a product manager does and what a product owner does.

For example, just after a company is founded, the CEO is presumably the best person to take care of the product roles. They are very often (at least in scalable SaaS companies) the one who wears the hat of PO, PM, QA & Support. It is not a choice or a strategy. It is just what the moment implies.

Also, in general, the way a company’s momentum is handled defines how “well” a company is doing.

Has the moment happening right now been anticipated, or is it just the consequence of the reaction of the last moment? The more a company reacts, the more “Agile” you can consider it to be. The more moments are anticipated, the more “visionary” you can consider the company. It’s challenging to be both.

2. Size matters

Size here doesn’t necessarily refer to the number of employees a company has. The size of a company could also refer to the number of:

products
models
features
scopes
price grids
or markets they address.

Cutting the products in the right “dimensions” and deciding if we split per market, per range or per topic can be a lifelong job, because it changes very often and the faster the company evolves, the more it has to be changed.

Size can also be defined by the number of users, clients, partners, retailers, languages, unit systems,… But also and more frequently, a combination of all the above will tell how many PMs, POs and Heads of Product your organization will need.

For example, in many non-profits, product managers are based on the donor personas. You have a PM for small donation amounts (say $10 to $500), one for larger ones ($500 to $10,000) and one for those that are even bigger. Why? Because expectations from each persona type differ radically, and being able to cater to each will help the non-profit grow in the long run.

In short, the size of the company (its client portfolio, its product catalogue…) may define what the PM does, what the PO does, and how many of each there will be.

3. The remaining criteria

But these are not the only factors that define how many PMs, POs or any other product-facing people you will need. Many other factors can also be taken into account, including:

number of developers
number of designers
shipping velocity
technical stacks
tooling
product lifecycle
individual wishes
growth potential
existing need coverage ratio
legacy ratio
human resources policy
the market you address
the level of politics installed in the company.

There is some kind of global agreement (at least in SaaS businesses) on the fact that the split between PM and PO is based on: vision vs. operations, projection vs. immediacy, value vs. metrics. But both are sides of the same coin.

In short, no two organizations are identical. In other words, there’s no one set answer to the question of what a PM vs. PO should do. Some organizations will need many, some will need none. Enabling people to work successfully together is hard enough. Copying and pasting a method won’t help – you have to find your own.

What skills are needed to be a good product manager and/or product owner?

Both product managers and product owners need a good balance of hard and soft skills to carry out their tasks efficiently.

The necessary skills for product managers include:

Excellent communication and collaboration skills
A good amount of technical expertise
Prioritization skills
Good business acumen
Analytical skills

Meanwhile, the necessary skills for product owners include:

Problem solving
Project management skills
Great communication and strong storytelling skills
Deep understanding of user data and analytics

The skills necessary for product managers and product owners may overlap especially in smaller companies where a product manager could also be a product owner and vice versa.

What KPIs are used to measure success for product owners and product managers?

It depends and may vary from one organization to the other depending on their objectives

However, if we tie them into their individual responsibilities, we can give a general overview of the kind of KPIs that can be used to measure their performance.

A product manager, for example, has a wide range of responsibilities including the creation and prioritization of the product roadmap along with cross-team collaboration to ensure alignment around the roadmap.

In that sense, a product manager’s performance is primarily related to product success in line with the overall business goals. Thus, usually product managers’ KPIs should be based on business metrics such as growth, revenue, churn rate and costs.

Meanwhile, product owners have a narrower role and based on the responsibilities we outlined above, their success is measured using KPIs based on delivery, quality, and internal team satisfaction.

Conclusion

A product owner and manager are great assets to have but having both or two separate people taking on these roles is not a must for every organization.

At the end of the day, the question is not whether you should have both a product manager and product owner but whether having both of those roles is right for your organization. You will have to look inward to understand what your business actually needs.

How you decide to structure your teams will depend on the processes you have in place and the kind of outcomes you’re hoping to achieve based on business objectives, customer and company needs.

While we give an overall idea of each role, these responsibilities are not set in stone and they could widely vary on a case by case basis.

What really matters is that you have the right people with the right skills working towards a shared goal, which is building a product customers will love and meets their requirements and needs.

You might also like...

See all

Article

2min read

How We Made AB Tasty’s Feature Experimentation SDK OpenFeature-Compatible — And Even Easier to Adopt

Julie Dumont

Apr 30, 2025

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Subscribe to
our Newsletter

Article

Apr 27, 2023

13min read

How to Deal with Low Traffic in CRO

Hubert Wassner

If your website traffic numbers aren’t as high as you may hope for, that’s no reason to give up on your conversion rate optimization (CRO) goals.

By now you must have noticed that most CRO advice is tailored for high-traffic websites. Luckily, this doesn’t mean you can’t optimize your website even if you have lower traffic.

The truth is, any website can be optimized – you just need to tailor your optimization strategy to suit your unique situation.

In this article, we will cover:

The most common complaint about CRO (95% threshold and where it comes from)
An appropriate threshold for low traffic
Ideas on how to optimize your website with a smaller traffic flow
The CUPED testing technique
When CUPED works and doesn’t work

CRO analogy

In order to make this article easier to understand, let’s start with an analogy. Imagine that instead of measuring two variants and picking a winner, we are measuring the performance of two boxers and placing bets on who will win the next 10 rounds.

So, how will we place our bet on who will win?

Imagine that boxer A and boxer B are both newbies that no one knows. After the first round, you have to make your choice. In the end, you will most likely place your bet on the boxer who won the first round. It might be risky if the winning margin is small, but in the end, you have no other way to base your decision.

Imagine now that boxer A is known to be a champion, and boxer B is a challenger that you don’t know. Your knowledge about boxer A is what we would call a prior – information you have before that influences your decision.

Based on the prior, you will be more likely to bet on boxer A as the champion for the next few rounds, even if boxer B wins the first round with a very small margin.

Furthermore, you will only choose boxer B as your predicted champion if they win the first round by a large margin. The stronger your prior, the larger the margin needs to be in order to convince you to change your bet.

Are you following? If so, the following paragraphs will be easy to grasp and you will understand where this “95% threshold” comes from.

Now, let’s move on to tips for optimizing your website with low traffic.

1. Solving the problem: “I never reach the 95% significance”

This is the most common complaint about CRO for websites with lower traffic and for lower traffic pages on bigger websites.

Before we dig into this most common problem, let’s start by answering the question, where does this 95% “golden rule” come from?

The origin of the 95% threshold

Let’s start our explanation with a very simple idea: What if optimization strategies were applied from day one? If two variants with no previous history were created at the same time, there would be no “original” version challenged by a newcomer.

This would force you to choose the best one from the beginning.

In this setting, any small difference in performance could be measured for decision-making. After a short test, you will choose the variant with the higher performance. It would not be good practice to pick the variant that had lower performance and furthermore, it would be foolish to wait for a 95% threshold to pick a winner.

But in practice, optimization is done well after the launch of a business.

So, in most real-life situations, there is a version A that already exists and a new challenger (version B) that is created.

If the new challenger, version B, comes along and the performance difference between the two variants is not significant, you will have no issues declaring version B “not a winner.”

Statistical tests are symmetric. So if we reverse the roles, swapping A and B in the statistical test will tell you that the original is not significantly better than the challenger. The “inconclusiveness” of the test is symmetric.

So, why do you set 100% of traffic toward the original at the end of an inconclusive test, implicitly declaring A as a winner? Because you have three priors:

Version A was the first choice. This choice was made by the initial creator of the page.
Version A has already been implemented and technically trusted. Version B is typically a mockup.
Version A has a lot of data to prove its value, whereas B is a challenger with limited data that is only collected during the test period.

Points 1 & 2 are the bases of a CRO strategy, so you will need to go beyond these two priors. Point 3 explains that version A has more data to back its performance. This explains why you trust version A more than version B. Version A has data.

Now you understand that this 95% confidence rule is a way of explaining a strong prior. And this prior mostly comes from historical data.

Therefore, when optimizing a page with low traffic, your decision threshold should be below 95% because your prior on A is weaker due to its traffic and seniority.

The threshold should be set according to the volume of traffic that went through the original from day one. However, the problem with this approach is that we know that the conversion rates are not stable and can change over time. Think of seasonality – i.e. black Friday rush, vacation days, Christmas time increases in activity, etc. Because of the seasonal changes, you can’t compare performances in different periods.

This is why practitioners only take into account data for version A and version B taken at the same period of time and set a high threshold (95%) to accept the challenger as a winner in order to formalize a strong prior toward version A.

What is the appropriate threshold for low traffic?

It’s hard to suggest an exact number to focus on because it depends on your risk acceptance.

According to the hypothesis protocol, you should structure a time frame for the data collection period in advance.

This means that the “stop” criteria of a test are not a statistical measure or based on a certain number. The “stop” criteria should be a timeframe coming to an end. Once the period is over, then you should look at the stats to make an appropriate decision.

AB Tasty, our customer experience optimization and feature management software, uses the Bayesian framework which produces a “chances to win” index which encourages a direct interpretation instead of a p-value, which has a very complex meaning.

In other words, the “chances to win index” is the probability for a given variation to be better than the original.

Therefore, a 95% “chance to win” means that there is a 95% probability that the given variation will be the winner. This is assuming that we don’t have any prior knowledge or specific trust for the original.

The 95% threshold itself is also a default compromise between the prior you have on the original and a given level of risk acceptance (it could have even been a 98% threshold).

Although it is hard to give an exact number, let’s make a rough scale for your threshold:

New A & B variations: If you have a case where variation A and variation B are both new, the threshold could be as low as 50%. If there is no past data on the variations’ performance and you must make a choice for implementation, even a 51% chance to win is better than 49%.
New website, low traffic: If your website is new and has very low traffic, you likely have very little prior on variation A (the original variation in this case). In that case, setting 85% as a threshold is reasonable. Since it means that if you put aside the little you know about the original you still have 85% to pick up the winner and only 15% to pick a variation that is equivalent to the original, and a lesser chance that it performs worse. So depending on the context, such a bet can make sense.
Mature business, low traffic: If your business has a longer history, but still lower traffic, 90% is a reasonable threshold. This is because there is still little prior on the original.
Mature business, high traffic: Having a lot of prior, or data, on variation A suggests a 95% threshold.

The original 95% threshold is far too high if your business has low traffic because there’s little chance that you will reach it. Consequently, your CRO strategy will have no effect and data-driven decision-making becomes impossible.

By using AB Tasty as your experimentation platform, you will be given a report that includes the “chance to win” along with other statistical information regarding your web experiments. A report from AB Tasty would also include the confidence interval on the estimated gain as an important indicator. The boundaries around the estimated gain are also computed in a Bayesian way, which means it can be interpreted as the best and the worst scenario.

The importance of Bayesian statistics

Now you understand the exact meaning of the well-known 95% “significance” level and are able to select appropriate thresholds corresponding to your particular case.

It’s important to remember that this approach only works with Bayesian statistics since frequentist approaches give statistical indices (such as p-Values and confidence intervals that have a totally different meaning and are not suited to the explained logic).

2. Are the stats valid with small numbers?

Yes, they are valid as long as you do not stop the test depending on the result.

Remember the testing protocol says once you decide on a testing period, the only reason to stop a test is when the timeframe has ended. In this case, the stat indices (“chances to win” & confidence interval) are true and usable.

You may be thinking: “Okay, but then I rarely reach the 95% significance level…”

Remember that the 95% threshold doesn’t need to be the magic number for all cases. If you have low traffic, chances are that your website is not old. If you refer back to the previous point, you can take a look at our suggested scale for different scenarios.

If you’re dealing with lower traffic as a newer business, you can certainly switch to a lower threshold (like 90%). The threshold is still higher because it’s typical to have more trust in an original rather than a variant because it’s used for a longer time.

If you’re dealing with two completely new variants, at the end of your testing period, it will be easier to pick the variant with the higher conversions (without using a stat rest) since there is no prior knowledge of the performance of A or B.

3. Go “upstream”

Sometimes the traffic problem is not due to a low-traffic website, but rather the webpage in question. Typically, pages with lower traffic are at the end of the funnel.

In this case, a great strategy is to work on optimizing the funnel closer to the user’s point of entry. There may be more to uncover with optimization in the digital customer journey before reaching the bottom of the funnel.

4. Is the CUPED technique real?

What is CUPED?

Controlled Experiment Using Pre-Experiment Data is a newer buzzword in the experimentation world. CUPED is a technique that claims to produce up to 50% faster results. Clearly, this is very appealing to small-traffic websites.

Does CUPED really work that well?

Not exactly, for two reasons: one is organizational and the other is applicability.

The organizational constraint

What’s often forgotten is that CUPED means Controlled experiment Using Pre-Experiment Data.

In practice, the ideal period of “pre-experiment data” is two weeks in order to hope for a 50% time reduction.

So, for a 2-week classic test, CUPED claims that you can end the test in only 1 week.

However, in order to properly see your results, you will need two weeks of pre-experiment data. So in fact, you must have three weeks to implement CUPED in order to have the same accuracy as a classic 2-week test.

Yes, you are reading correctly. In the end, you will need three weeks time to run the experiment.

This means that it is only useful if you already have two weeks of traffic data that is unexposed to any experiment. Even if you can schedule two weeks of no experimentations into your experimentation planning to collect data, this will be blocking traffic for other experiments.

The applicability constraint

In addition to the organizational/2-week time constraint, there are two other prerequisites in order for CUPED to be effective:

CUPED is only applicable to visitors browsing the site during both the pre-experiment and experiment periods.
These visitors need to have the same behavior regarding the KPI under optimization. Visitors’ data must be correlated between the two periods.

You will see in the following paragraph that these two constraints make CUPED virtually impossible for e-commerce websites and only applicable to platforms.

Let’s go back to our experiment settings example:

Two weeks of pre-experiment data
Two weeks of experiment data (that we hope will only last one week as there is a supposed 50% time reduction)
The optimization goal is a transaction: raising the number of conversions.

Constraint number 1 states that we need to have the same visitors in pre-experiment & experiment, but the visitor’s journey in e-commerce is usually one week.

In other words, there is very little chance that you see visitors in both periods. In this context, only a very limited effect of CUPED is to be expected (up to the portion of visitors that are seen in both periods).

Constraint number 2 states that the visitors must have the same behavior regarding the conversion (the KPI under optimization). Frankly, that constraint is simply never met in e-commerce.

The e-commerce conversion occurs either during the pre-experiment or during the experiment but not in both (unless your customer frequently purchases several times during the experiment time).

This means that there is no chance that the visitors’ conversions are correlated between the periods.

In summary: CUPED is simply not applicable for e-commerce websites to optimize transactions.

It is clearly stated in the original scientific paper, but for the sake of popularity, this buzzword technique is being misrepresented in the testing industry.

In fact, and it is clearly stated in scientific literature, CUPED works only on multiple conversions for platforms that have recurring visitors performing the same actions.

Great platforms for CUPED would be search engines (like Bing, where it has been invented) or streaming platforms where users come daily and do the same repeated actions (playing a video, clicking on a link in a search result page, etc).

Even if you try to find an application of CUPED for e-commerce, you’ll find out that it’s not possible.

One may say that you could try to optimize the number of products seen, but the problem of constraint 1 still applies: a very little number of visitors will be present on both datasets. And there is a more fundamental objection – this KPI should not be optimized on its own, otherwise you are potentially encouraging hesitation between products.
You cannot even try to optimize the number of products ordered by visitors with CUPED because constraint number 2 still holds. The act of purchase can be considered as instantaneous. Therefore, it can only happen in one period or the other – not both. If there is no visitor behavior correlation to expect then there is also no CUPED effect to expect.

Conclusion about CUPED

CUPED does not work for e-commerce websites where a transaction is the main optimization goal. Unless you are Bing, Google, or Netflix — CUPED won’t be your secret ingredient to help you to optimize your business.

This technique is surely a buzzword spiking interest fast, however, it’s important to see the full picture before wanting to add CUPED into your roadmap. E-commerce brands will want to take into account that this testing technique is not suited for their business.

Optimization for low-traffic websites

Brands with lower traffic are still prime candidates for website optimization, even though they might need to adapt to a less-than-traditional different approach.

Whether optimizing your web pages means choosing a page that’s higher up in the funnel or adopting a slightly lower threshold, continuous optimization is crucial.

Want to start optimizing your website? AB Tasty is the best-in-class experience optimization platform that empowers you to create a richer digital experience – fast. From experimentation to personalization, this solution can help you activate and engage your audience to boost your conversions.

You might also like...

See all

Article

2min read

How We Made AB Tasty’s Feature Experimentation SDK OpenFeature-Compatible — And Even Easier to Adopt

Julie Dumont

Apr 30, 2025

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Subscribe to
our Newsletter

Article

Apr 25, 2023

10min read

Why You Should Slot Feature Flags into Your Agile Roadmap

AB Tasty

It’s easy to lose your way when building an Agile roadmap.

If you get too detailed with your planning, you end up building a roadmap that is Agile in name alone but looks more like a traditional Waterfall roadmap. If you don’t perform enough planning, then you’ll produce a skeleton of a roadmap that sends you running in multiple directions without ever arriving anywhere meaningful.

The correct approach lies somewhere in the middle. You keep things loose, nimble, and iterative but you also set a beacon that will guide each of your sprints to an impactful destination.

From our experience, one “beacon” that will keep your Agile product roadmap grounded, and your products moving in the right direction, is a simple function— the feature flag.

It isn’t fancy. It isn’t flashy. And it doesn’t look overly strategic. If you use feature flags properly, then they will keep your Agile roadmap focused on the outcomes that matter most without forcing you down a fixed path. Here’s why.

First principles: The real benefit of Agile over Waterfall

It feels like a given these days: if you work as a Product Manager (especially in the tech sector) then you’re going to follow some kind of Agile methodology. Depending on your work history, you may never have worked with a Waterfall roadmap, let alone developed one, in your entire career.

If that’s the case, it might even feel confusing why Waterfall was ever developed. The methodology is slow. It’s rigid. It’s opaque. On the surface, it looks inferior to Agile in every way. But once you dig into it a little, there is one area where waterfall trumps Agile. Waterfall is a better fit within a traditional corporate context than Agile.

While Agile and Waterfall are popular in software development, each one is best suited for different types of projects.

For example, a Waterfall approach makes sense when a software project has clearly defined requirements with low probability that any changes will occur halfway through.

Waterfall fits really well into that broader corporate world’s standard operating procedures. It collects business requirements in a standard one-off phase and then sets them in stone with a concrete project. Waterfall adopts a more linear way of working so that development phases flow in one direction just like the flow of a waterfall, hence the name and tends to occur over a long period of time.

It breaks that project into a clear, crisply defined plan and each step must be completed before moving onto the next phase. In the end, the project’s success will be defined by how well its leaders completed the milestones in the project’s plan, and if they delivered to the project’s requirements on-time and on-budget.

Waterfall methodology isn’t really about trying to create the most effective, efficient, or accountable system. It’s about having the product developers and managers operate in a way that makes sense to a large, lumbering corporation.

A new approach—Agile— was only possible because it was developed outside of this legacy corporate context. In fact, Agile is an iterative approach that came about as a response and alternative to Waterfall’s rigid and linear structure.

And here’s what they came up with: product management would deliver a greater impact if it stopped lining up to what a corporation wanted, and if it instead lined up to what actual real-world users want.

In an Agile approach, which introduces flexibility, teams work on multiple phases at the same time with the goal to enable faster software delivery for the purpose of collecting customer feedback. It does this by breaking down the software development life cycle into sprints, which could last from one to four weeks, that include regular feedback loops.

Incremental releases means teams can build better features much quicker that offer more value and optimize and iterate these features based on the feedback received. It aligns the product not only with the product vision but also with customer needs.

This is the real innovation of an Agile roadmap over a Waterfall one. It isn’t the increased speed & efficiency that everyone fixates on. It’s the simple but powerful fact that an Agile roadmap re-aligns the product manager’s focus onto the user.

Here are some of the advantages of an Agile methodology:

Faster feedback loops
Higher customer satisfaction
Reduced time-to-market
Increased flexibility with more room for innovation
Enhanced productivity by breaking down projects into smaller, more manageable chunks

And most of Agile methodology’s core user-alignment activities occur during Feature Release Management and are brought to life by the right feature flag tool.

A quick caveat: Yes, business impact still matters in Agile

Before we move on, let’s make one point very clear.

When we say Waterfall aligns well to corporate context, we mean corporate operational context. But we don’t mean a Waterfall approach offers the best way to deliver results.

Most often, these big Waterfall projects deliver poor results because they can take months—or even years—between their initial requirements collection and their project’s completion. During this time, the project’s alignment, and even its viability to its users, often shifts, reducing its chances of producing any meaningful business impact.

By contrast, a properly developed and managed Agile roadmap will maintain alignment with its users throughout its entire lifecycle and deliver concrete, measurable, and accountable results.

Feature release management, and feature flags, can also drive this tight connection between user-centered development and KPI improvement. We’ll get to how in just a minute.

Feature release management: The heart of any effective Agile roadmap

From a user-alignment perspective, feature releases are the key point that differentiates an Agile roadmap from a Waterfall roadmap.

Agile looks different from Waterfall in many areas of activity.

In Waterfall, new products and features are released to all users at once, in a single big bang, after a very long development cycle. In an Agile roadmap, new products and features can be—and should be—released at a much faster rate.

This is the key functional difference that makes Agile more user-centered than Waterfall. Rapid and effective feature release management lets you:

Keep your users top-of-mind at all times.
Regularly collect your users’ data and feedback.
Use up-to-date feedback to guide your development cycles.
Repeat the cycle, to make sure you correctly incorporated user feedback in your next round of features and product updates.

If you want to keep your development user-centered then feature release management is critical to effectively incorporate into your Agile product roadmap. Here’s how.

The 5 key elements to include in your Agile release planning

Agile release planning is key to building customer-centric products by allowing you to prioritize and release product requirements as needed. In other words, it allows you to plan your product’s incremental releases- your features- and helps ensure your project is headed in the right direction and following the Agile methodology.

It differs from a product roadmap in that release planning focuses on one sprint at a time (on short-term goals) while a product roadmap looks further ahead in the future and focuses on long-term objectives.

Put simply, the goal of a release plan is to help you prioritize features of your product and focus on releasing specific features in less time to improve the customer experience. Thus, teams use this kind of planning when they’re dividing a project into short sprints or increments instead of planning for one major product release.

It is a unique approach to planning as it takes into account the flexible nature of software development by leaving room for any necessary adjustments as you go through the development lifecycle to incorporate customer (and stakeholder) feedback.

The idea is to be open to prioritizing tasks to provide improved value to your customers.

Here are the key elements to include in each of your feature releases that will turn them into a critical, recurring touchpoint between you and your users.

1. User segmentation

At a basic level, you need to carefully select which user audiences you will first release (and test) new features and products to.

At a deeper level, user segmentation can flow throughout every step of feature release management. You can personalize the experience of your new products and features to each segment you test them with. In other words, you try out different versions of each new product or feature with different segments.

During testing, you can rapidly toggle features off for segments who are not responding well to them. And you can even guide the ongoing development of your products and features depending on which user segments respond the best to them.

2. KPIs measurement

However you measure product or feature success, you must quantify it, and measure those metrics in real-time during each release.

Doing so serves two purposes: First, it gives you the ability to produce an accurate, objective measure about which products and features are succeeding with which segment (and whether or not you are actually improving their performance during each development sprint).

Second, they let you demonstrate concrete, measurable, and accountable results for your business—to both report on the success of your most recent development, and to create meaningful justifications for more robust rollouts.

3. Governance

You need some formalized way to make decisions off the data that you produce. When do you toggle a feature on or off and for who? When do you roll out the product or feature to new segments? When is a product or feature ready to deploy to your entire user community?

To make these decisions, you must have established definitions for success (see “KPIs”), and defined procedures for monitoring and acting on release performance data both in real-time and during post-release recaps.

4. A/B testing

Any time you are segmenting audiences, testing multiple variations on products and features, and collecting copious amounts of real-world user data, then you are setting the stage for multiple A/B tests.

By performing comprehensive A/B tests during each of your feature releases, you will eliminate multiple developmental dead ends and narrow the list of viable “next steps” for your next sprint.

5. Automation

If you incorporate these four elements, then your feature release management process will get pretty complex, pretty quickly. But if you select the right tool to automate as many of these elements and their internal processes,as possible, then you can let go of most operational elements. Afterwards, you would simply sit back during feature releases and focus on making informed decisions before, during, and after each of your releases.

By incorporating each of these five elements into your feature release process, you will ensure that each of these critical touch points brings you and keeps you as close as possible to your users.

And, thankfully, there is one single function that incorporates each of these elements and makes them a practical and effortless habit in your Agile roadmap— feature flags.

Bringing it all home: Feature flags

At their core, the goal of feature flags is to enable you to toggle features on or off, with a single click on your dashboard, without having to adjust your codebase.

That may seem very basic at first glance but buried in this simplicity is a lot of depth, and a lot of room to easily deliver on each of the above elements of user-centered feature release management.

With the right feature flag tool, you can:

Perform sophisticated real-time control over which user segments get new products and features.
Attach core KPIs to your releases and immediately kill products and features that are not performing well while immediately expanding the release of those that are knocking it out of the park.
Monitor your results (and take action) in real-time.
Easily manage and act on complex A/B tests.
Bundle feature flags in with a complete suite of feature release functionality to wrap the whole exercise up in a single, highly-automated platform.

We kept each of these functions in mind when we built our own Feature Flag function, and release management platform.

If you’d like to take it for a test run and see how easy you can incorporate the core actions of Feature Flagging, feature release management, and user-centered Agile product management into your roadmap, drop us a line!

You might also like...

See all

Article

2min read

How We Made AB Tasty’s Feature Experimentation SDK OpenFeature-Compatible — And Even Easier to Adopt

Julie Dumont

Apr 30, 2025

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Subscribe to
our Newsletter

Article

Apr 12, 2023

4min read

AB Tasty Unleashes New GA4 Integration for Next-Level Experimentation

John Hughes

We know how important data is to our clients. Understanding what people are doing on your website, app, or mobile site, is vital for businesses to create the best digital experiences. To meet customer demands for a more personalized experience, brands turn to AB Tasty to optimize at every stage of the digital customer journey.

And now we’re taking your data-driven optimization strategy even further with our GA4 integration.

What this integration means for marketing teams

By connecting AB Tasty and Google Analytics 4 (GA4), marketing teams have a clearer vision of how visitors interact with their site via advanced analytics on CPA, conversion rate, bounce rate, SEO and traffic. This integration means you can use data from either tool to better understand the effects of your experimentation during or post-rollout and drive more innovative ideas with data-backed hypotheses.

With this simplified integration, you can seamlessly connect Analytics with AB Tasty enabling you to leverage the robust reporting and intelligence features of Analytics, while taking advantage of our digital experience software.

The AB Tasty and Google collaboration offers businesses a powerful toolkit for optimizing their digital experiences and driving better results. By combining advanced analytics capabilities with sophisticated testing and personalization features, businesses can gain a competitive edge and deliver more value to their customers, deepen their understanding of users’ behavior and preferences, and use that knowledge to create more effective and engaging experiences.

With the ability to test and personalize every element of their digital experiences, businesses can ensure that their websites, apps, and campaigns are optimized for maximum engagement and conversion, ultimately leading to increased revenue and customer loyalty. In short, the AB Tasty and Google integration is essential for businesses looking to stay ahead of the curve in today’s fast-paced digital landscape.

AB Tasty partners with Google Cloud

As AB Tasty strengthens its partnership with Google, we are delighted to announce that we are an official Google Cloud Partner.

This means that clients and partners can access AB Tasty’s digital experience tools and Google’s secure platform. Engage and activate your clients through AB Tasty’s suite of products, testing and experimentation, personalization, server-side testing, feature flags, search and product recommendations, to create resilient digital products, make exceptional customer experiences, and deliver strong business results.

A continued partnership

This strategic alliance is crucial for businesses looking to offer exceptional products and services to their customers. By harnessing the advanced capabilities of AB Tasty’s platform and Google Cloud’s cutting-edge technologies in AI and ML, this partnership promises to empower companies in delivering optimized solutions that drive superior outcomes for their consumers.

Businesses with an annual commitment to Google Cloud can easily add AB Tasty solutions to their packages and quotes with ease. The partnership will also enable customers to access existing vendor relationships with Google Cloud, streamlining the deployment of AB Tasty on the platform.

Google Optimize Sunset

AB Tasty is excited to be a launch partner for Google Analytics newly released experimentation dimension and to help Google Optimize customers transition to new software.

By making the move to AB Tasty, you can gain access to top-of-the-line experience optimization tools to elevate your digital experiences to the next level. For companies that have progressed further in their CRO journey and require more extensive experimentation capabilities, AB Tasty provides a superior solution.

Discover the perks of our advanced solution that delivers a range of integrations with diverse providers, personalized features, pre-built widgets for optimizing the customer journey, and expert CSMs and account managers to offer personalized support throughout the entire contract period. This includes seamless transfer of your test history and data from Google Optimize.

Improving each touchpoint for your customers and learning from every product and feature rollout can have a big impact on your ROI.

The GA4 and AB Tasty integration levels up your ability to create highly personalized experiences for each step of the customer journey with a wide range of audience targeting options and in-depth data on user behavior.

Are you ready to make the move?

The post-Google Optimize world doesn’t have to be bleak.

As one of Google’s top picks as your new A/B testing platform, AB Tasty is a best-in-class A/B testing tool that helps you convert more customers by leveraging experimentation to create a richer digital experience – fast. This experience optimization platform embedded with AI and automation can help you achieve the perfect digital experience with ease.

You might also like...

See all

Article

2min read

How We Made AB Tasty’s Feature Experimentation SDK OpenFeature-Compatible — And Even Easier to Adopt

Julie Dumont

Apr 30, 2025

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Subscribe to
our Newsletter

Article

Apr 6, 2023

11min read

How Can Teams Use Feature Flags in Mobile App Deployment?

Rowan Haddad

In the digital age, companies can no longer only focus their efforts on optimizing for desktop especially as more and more consumers are using their mobile devices to visit websites and make purchases through apps.

However with millions of apps out there, competition and consumer demands and expectations are at an all-time high. This means your app needs to stand out in an overcrowded market.

It’s important to point out that deploying mobile apps doesn’t follow the same process as a website app.

In this article, we will investigate the challenges of mobile app deployment and release and how feature flags are the key to help you deliver optimized mobile apps that meet your consumers’ needs.

The challenges of mobile app deployment

Mobile development teams are particularly susceptible to bugs and long, tedious release cycles.

In short, there are two main problems when it comes releasing or updating features on mobile apps:

You have to wait until approval from app stores (which could take some time and significantly delay a release).
Afterwards, you have to wait for users to manually download the update from the store (which could also take a lot of time).

For example, let’s take a look at this scenario: you’re working on an update to your mobile app. You finally release it only to find out that there’s a bug you missed that’s causing your app to crash.

By the time you release a new update with a fix to the issue, wait for the release to the app store and watch for users to download the update, you might risk losing a significant number of users.

Mobile developers and engineers are all too familiar with such a scenario.

Therefore, it can be a painstakingly long process to get a release approved. Once approved, any buggy release will need to be fixed and put through the app store approval process all over again, leading to further delays.

Although the review time has improved in recent years, if your app fails to meet the app store review guidelines it may be further delayed. This means that you cannot push real-time updates to production as you would with web apps.

Put simply, the process of deploying mobile apps is not as straightforward as it might be for web apps.

Unlike web apps which are automatically updated once visitors access the site, users have to download an update of the mobile app in their store to get the latest version. As updates pile up after going through the review process, you have no control over whether users download the latest versions.

Therefore, it can take more time to deploy mobile app updates compared to web apps. And in an age where customers demand the best every time, it’s not feasible to have them wait that long for an update, especially when there’s a bug involved, much less having to wait to deploy a new app version once the bug is removed.

In modern software development, when continuous delivery is vital to retain competitiveness and meet fast-changing consumer demands, teams must turn to another solution to achieve a more frequent release cadence.

The value of feature flags in mobile app deployment and release

This is where feature flags come into play.

Unlike client-side testing where experiments are focused on web browsers, feature flags give teams the ability to carry out server-side experiments across multiple channels including mobile apps.

Feature flags allow teams to enable or disable features to users of their choosing to minimize risk and negative impact.

This is because feature flags allow you to decouple deployment from feature releases meaning you can turn functionality on or off remotely without redeploying code to app stores and waiting for its approval or having to wait for all changes to be ready at the same time to release your own changes. This way you can deploy code to whoever whenever you want.

Thus, you can upgrade your app continuously in real time based on feedback from your users without sending an app store update or waiting on its approval. You can also gradually release new features without users having to go through the hassle of always having to update their app version.

With feature flags, mobile developers can safely test in production on a pre-defined audience and disable any features with a kill switch should any issues come up, thereby reducing any negative impact. Developers can then work on pinpointing the issue and fixing it before releasing the feature to all users.

How can you use feature flags in mobile apps?

Feature flags can be used not only by developers but also by product and release managers to optimize mobile experiences in various ways.

Here are some examples of use cases for mobile feature flags:

A/B testing: With feature flags, you can divide your users into subsets with each set of users receiving a different variation of the feature. This allows you to test and determine which is the top-performing variation to roll out to all your users. Put simply, running A/B tests allow you to collect valuable live feedback from your users so you can make informed decisions about how to optimize your features and products.
Targeted rollouts: Teams can use feature flags to test out their ideas by progressively rolling out their feature, giving only a limited number of users ]early access to the app through beta testing for example. This helps generate buzz around your release and lets you monitor its impact on these select users. Targeted rollouts allow teams to make more informed decisions about what to optimize and fine-tune an app based on live user feedback.
Personalization: Feature flags are a great way to personalize experiences for different types of users rather than delivering a unified experience for all your users. By changing the features that certain users receive, you can tailor the user experience in mobile apps to individual users or user segments. For example, you can offer a unique experience based on the country the user is in.
Roll back/kill switch: What’s really unique about feature flags is that they enable teams to roll back any buggy updates quickly. By simply disabling the relevant feature flag, you can solve a bug without going through the lengthy app store review process.

Mobile feature flags: Use cases

We’ve talked mainly about how feature flags can be used in mobile app deployment but they’re also a great way to reduce risk when deploying and testing on mobile sites, especially when it comes to deep level modifications tied to back-end architecture such as testing new payment methods.

This can be easily done using a feature flagging platform, where teams can safely deploy frequent releases with an easy-to-use platform that can be used by all teams.

For example, let’s say you developed two payment features: one for desktop and one for mobile. Before doing a full release, you’re looking to test them on a small group of early adopters to monitor its impact and determine its usage rate.

Using AB Tasty, you can easily create a Feature Toggling use case on your AB Tasty account and choose the KPI you want to follow, in this case that would be the clicks on the “Proceed to checkout” button and then “conversion rate” as a sub KPI.

You can then define the two scenarios: one to enable the feature on desktop and another to enable it on mobile devices. You will then configure the flag that will turn on the new payment method for each scenario as seen in the image below in the “Scenario mobile” on the dashboard.

Next, let’s take a look at real-life examples of how AB Tasty clients use feature flags to carry out mobile testing:

Use case 1

Decathlon, a French sporting goods retailer with over 2,000 stores in 56 countries, wanted to test CTA placement to measure its impact across all devices, including mobile, and product listing pages (PLPs) with the help of feature flags.

In the original version, seen below, Decathlon APAC’s team was looking to test an earlier placement of the “add to cart” button on mobile on the main page below the product image to ensure a positive rollout and measure uplift. In the original version, users had to click on the product to go to the PDP before seeing this button.

With AB Tasty’s robust solution, the team was able to test the impact of this new feature on conversion. Changing the CTA placement proved to be a success, resulting in a 10.37% increase in transaction rate and an $11.27 increase in the average order value.

Use case 2

EDF (Electricité de France) is the largest electricity supplier in France for over 70 years. The team at EDF wanted to increase the number of online subscriptions and calls via their application.

In particular, they wanted to monitor the effect of changing the CTA design in the app. Using feature flags to increase the visibility of the CTAs, the team could then measure the impact on (and boost) clicks for online subscriptions and/or calls with EDF advisors respectively.

The team ran an A/B test with the subscription CTA against an orange background and the call CTA against a green background. They also added text to communicate hours of operation.

The call CTA was the one that yielded more positive results allowing the team to generate more qualified leads with an increase in calls with EDF advisors.

Thus, with a 20% increase in calls, the team could then confidently develop and roll out an adapted variation in the app where the new call CTA was more visible.

Use case 3

Often, A/B tests are a fool-proof way to eliminate potential bias and can save a company from investing in an optimization campaign that would otherwise take up a lot of valuable time and resources.

That was the case with Ornikar, a driving school platform in France with more than 2.5 million customers. The team was looking to revamp the home screen of their application and needed to identify which changes should be kept and which should be discarded.

The team set up an A/B test on AB Tasty to replace the existing carousel of four slides and two CTAs (left image) with a new screen featuring Ornikar benefits, a new CTA order and a more detailed carousel (right image).

The test was conducted for a duration of three weeks. After a week, they found that the new variation was not performing as wellas they expected so the team paused the test and adjusted the CTA and ran the test again for two weeks.

The results were still negative after two weeks and the team decided not to deploy the new home screen into production.

Due to the flexibility of the AB Tasty platform, the team was able to make quick iterations over a short period of time. Above all, Ornikar was able to avoid losing out on conversions and wasting time and resources as well as minimizing negative impact by testing the new home screen first before rolling it out to all its users.

Feature flags: The ultimate tool for better mobile experiences

As we’ve seen, feature flags are a powerful tool allowing teams across an organization to have more control over mobile testing and release while reducing risk.

Beyond giving you full control of new feature releases despite App and Play Store approval processes, feature flags enable teams to optimize their mobile apps and personalize the user experience. They also allow you to ship features more often and obtain quick feedback from your most relevant users.

With increasing mobile usage and millions of mobile apps to compete with, it’s essential to provide the very best user experience on mobile devices. Running experiments and using progressive rollouts with feature flags are key to delivering optimal and great mobile experiences.

Using a third-party feature flagging platform makes it easy to toggle features on and off and remotely configure your flags straight from the interface. By controlling all your feature flags in an easy to use web dashboard, it also ensures you’re keeping up with essential best practices to set you up for success and help you stand out from competitors.

You might also like...

See all

Article

2min read

How We Made AB Tasty’s Feature Experimentation SDK OpenFeature-Compatible — And Even Easier to Adopt

Julie Dumont

Apr 30, 2025

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Subscribe to
our Newsletter

Article

Mar 30, 2023

9min read

10 Best Practices for Shopping Cart Page Optimization

Anthony Brebion

Did you know that 70% of online shoppers abandon their carts?

According to the Baymard Institute, this number comes from an in-depth analysis that benchmarked 48 different studies on shopping cart abandonment rates.

This may not come as a surprise for experienced e-marketers, but the truth is that 7 people out of 10 will actually abandon their cart and end their shopping journey without making a purchase.

This is clearly a lot of lost revenue. So, how can you bring that percentage down?

This article aims to answer that question. We will cover best practices to help you diminish the abandonment rate and optimize your e-commerce shopping cart page for conversions.

Adopt clear UX parameters for shopping cart optimization

First of all, let’s start with a firm foundation. Your shopping cart page should adopt a clear, simple, and fast UX. This simple 3-step formula (CSF) is the cornerstone of any successful cart page:

Clear – There should be nothing messy, concerning, or misleading about your cart page. It should ideally display all the important information on a single page without the need to scroll too far or visit any other page.
Simple – Your cart page should display all the information using comprehensible, crystal-clear language and a design that leaves no room for misunderstanding.
Fast – The more time visitors spend on your cart page, the more likely they will leave it. If you apply the first two critical elements (simple+clear) to your page, the resulting cart page experience should also be fast.

As there are many elements on your page that you can optimize and run tests on to find the best solution, it’s important to follow this CSF framework for harmony.

Want to get started on A/B testing for your shopping cart page? AB Tasty is a great example of an A/B testing tool that allows you to quickly set up tests with low code implementation of front-end or UX changes on your web pages, gather insights via an ROI dashboard and determine which route will increase your revenue.

Knowing these 3 crucial elements, it’s high time we dive into our 10 best practices for e-commerce shopping cart pages.

10 best practices for your shopping cart pages

1. Create a detailed product summary

Just moments before your visitors proceed to checkout, they’ll land on your cart page which has one sole mission: lead your visitors to actually pay.

For most e-commerce buyers, the cart is a page used to review their order.

In order to help them do so, your mission is to clearly display all the relevant information regarding the product.

Below is a great example of a check-out page from ASOS that includes all the necessary details for a clear and easy review. Users know exactly which product they will purchase as well as the color, size, and quantity.

Detailed product summary on shopping cart page

When crafting your cart page, be sure to follow this example and include these elements:

Item thumbnail
Exact name
Item specifications (size, power, capacity, memory, features…)
Quantity of items in cart
Item color

Having all these elements shown to your customers allows them to quickly review their order and have confidence in their purchase.

Including all relevant details will decrease the percentage of cart abandonment that is typically caused by the lack of precise information.

2. Choose a clear, user-friendly color code

There have been many studies about the psychology behind colors. However, there’s no single answer on which color will fit all websites and solve all abandonment rate problems.

One thing that we do know for sure is that visitors love harmony and clear designs when it comes to UX.

Let’s look at the luxury shoe brand Jimmy Choo. For their check-out page, they kept a simple design by using a black button that clearly stands out, making it straightforward to click.

Clean color code on shopping cart page

Famous shoe-maker Jimmy Choo uses an elegant yet efficient black and white color code in order to clearly display the information on the cart page.

Notice that the checkout button efficiently stands out as the only black button on the page, making it extremely straightforward to click it.

3. Display explicit and detailed information about shipping and returns

What is the number one reason behind shopping cart abandonment in the US? Hidden shipping costs.
Online buyers despise hidden and last-minute shipping costs. As you can see in the image below from Baymard’s study, it’s by far the most popular reason for cart abandonment compared to others.

cart-abandonment-stats

Displaying transparent shipping and return policies is a key factor in enhancing conversion and gaining customers’ trust.

4. Craft clever information hierarchy and non-competing CTAs

Information hierarchy is the structure used to display and rank information according to its importance.

While designing cart pages, pay attention to the logic behind buttons, columns and titles as they will heavily influence the users’ perception.

You can use various colors in your CTAs (preferably matching your brand) although we recommend a maximum of 3-4 colors at a time.

Colors do help you gain visitors’ attention, so use them wisely:

Highlight important information
Use a distinct color to distinguish the CTA
Use lines or columns to structure your page

In the image below, Zappos, a USA-based shoe marketplace, does a great job of providing an efficient and clean shopping experience.

Shopping cart page with clear information hierarchy

The shopping cart page skillfully guides customers through the buying journey; we appreciate the neat layout emphasized by a simple 3-step color code (orange=very important; blue=important; grey=secondary).

5. Deploy payment options that your users love

Having different payment options is a necessity in today’s ultra-competitive e-commerce environment.

If you run an international e-commerce store, bear in mind that payment methods differ from one country to another: what’s used in North America isn’t necessarily the same as in Europe or in Asia.

To combat this, try to redirect customers based on their IP location to offer them a personalized experience based on the local currency and their preferred payment methods.

Various payment options on shopping cart page

In the image above, Adidas provides 6 different payment methods including Paypal, VISA and Apple Pay. This is an absolute necessity for large and global stores.

6. Show security seals and reassuring elements

In the same report from Baymard, the lack of trust in the payment accounted for 18% of abandoned cart rates. Trust seals, or trust badges, are very important to show your users that your site is legitimate.

In fact, they actually matter more than peer recommendations or trustworthy design. According to Baymard, here are the badges that give visitors the highest perception of a safe and secure site:

trust-seals-online-payments

7. Offer phone, chat or email assistance

Displaying a clear contact number and address details can impact your user’s level of trust. Shoppers want to make sure that your business is legitimate and not an online scam.

Furthermore, your visitors want to feel that there are actual humans behind your website.

Offering a live chat or phone assistance service right on the cart page is a great option to gain customers’ trust, legitimize your business and humanize your brand identity.

Wondering how to design your chatbox? Take a look at Victoria’s Secret’s page in the image below. They have chosen their most vibrant color, pink, to increase their call-to-action visibility.

Live chat integration on shopping cart

8. Add a “continue shopping” option

A “continue shopping” option is a smart way to offer your visitors a way to abandon their cart without leaving your website. They’ll have the freedom to continue browsing after they’ve already added items to their cart.

As some online shoppers actually use carts as “wishlists,” they can store items that they intend on purchasing later on.

9. Display legible thumbnail images

There’s nothing more annoying than a tiny thumbnail that barely helps to identify a product.

When customers review a product, you should give them the opportunity to see it correctly in a convenient size and resolution on both mobile and desktop.

Live chat integration on shopping cart

In the image above, Apple makes it very easy to distinguish the chosen product. The image used is bright and clear. Their customers will be absolutely certain that they’ve got the right item in their bag.

10. Push complementary products

Offering complementary product recommendations, or cross-selling, is nothing new in the e-commerce world.

However, displaying complementary products inside the shopping cart page is something worth testing if it could bring you a higher AOV.

For more testing ideas, check out our e-book: 50 Tests You Should Know For Website Optimization

Complementary products displayed on shopping cart page

Why optimize your shopping cart page?

An optimized shopping cart page is crucial to sales.

The shopping cart page is the last step your visitors take before their purchase. In this step of their digital user journey, it’s important to ease customer concerns in any way possible and promote a seamless checkout flow.

Every online e-commerce should be testing different elements of its cart page to find what works best for converting passive visitors into active customers.

Read more: Go one step further and improve your e-commerce product page performance to push your visitors to the cart page.

You might also like...

See all

Article

2min read

How We Made AB Tasty’s Feature Experimentation SDK OpenFeature-Compatible — And Even Easier to Adopt

Julie Dumont

Apr 30, 2025

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Subscribe to
our Newsletter

Article

Mar 20, 2023

8min read

Great Value Proposition Examples

Emily Healy

A value proposition is a clear statement of the fundamentals that define your brand.

To create a value proposition, you need to clearly define your product or service and showcase how you are different from competitors. This is often referred to as a Unique Selling Point (USP).

So, how can you define and present your company’s USP? By crafting a value proposition canvas.

A value proposition canvas is a visual display that maps out your product or service and how it can meet the specific needs of your customers.

It’s essentially an extension of the frequently-used business model canvas and has become a widely popular method of presentation in all corners of the business world.

In this article, we will dive into the specifics of a value proposition, give you tips for creating your own and advice on how to develop it further with a value proposition canvas.

What is a value proposition?

At first glance, a value proposition seems like a very simple and broad concept. However, a particularly effective value proposition is exceptionally complex and loaded with details.

Here are some common questions at the heart of each value proposition:

Why should a customer use your service or buy your product over a competitor’s?
What are you offering that no one else can?
How can you serve the needs of each individual customer?

A value proposition is not a positioning statement designed to appeal to as many customers as possible, rather it’s very specific. To meet the specific needs of a specific customer, you need to be as niche as possible.

For example, a business might be looking for software that comes with customer support packages that are more encompassing than usual due to the fact that they have little experience with a particular product.

Suppose a software company highlights its top-rated customer service team and affordable ongoing support options that their competitors are unable to provide. In that case, it will likely attract businesses that will profit from (and be in need of) this type of support.

Let’s look at another example: Uber – a travel service provider that meets low-cost and on-demand needs. Immediacy, convenience and simplicity are at the heart of their value proposition.

Uber value proposition

Think about the unique benefit that you are providing. If you can define this simply, concisely and accurately, you have a value proposition of worth to your audience.

How to create a value proposition

Now that we’ve defined what a value proposition is, let’s move on to how you can start brainstorming to transform your thoughts and ideas into a clear statement.

Before we get too intense, start by drafting up a few phrases with this template: We help (Customer) to (Goal) by (Unique value proposition).

We should also keep in mind that value propositions don’t need to be subtle. In fact, they can be rather blunt instruments used to draw in new customers or clients.

Value propositions should not be loaded with jargon or complexities. They should be thought of more like a well-researched advertising jingle than a dissertation.

Let’s discuss a few points to keep in mind when developing your value proposition:

Define your target audience
Know your competition
Tell your customers how you meet their needs
Be prepared to combat stereotypes

Define your target audience

As value propositions are highly dependent on those that are being sold to, it’s essential to define your target audience.

This requires a great deal of research, but will undoubtedly pay off in the long run. Age, demographics, marital status, economic status and personal interests can all inform how a value proposition is designed.

Your message needs to be clearly personalized for your audience.

Know the competition

It’s hard to stand out from the crowd if you have little concept of what the crowd is offering.

If a larger competitor offers the same USP (Unique Selling Points), it’s likely that they will get a hold of your potential customers since they will rely on your differences: brand size and reputation.

This requires extensive research to define new and fertile ground. You need to know your competition well – not only to be aware of your similarities but also to let your audience know how you stand out.

Tell your customer how you meet their needs

To start with, it’s best to define exactly what your product or service offers. Then, you can move into what makes it unique:

Customer service?
Affordability?
A simpler design than the competitors?

Once you have defined what you will highlight, it’s time to get more specific with your word choice. Remember: specificity does not mean complexity.

If the customer service is highly rated, why?
Affordability is great, but does the quality of the product match or even improve on your competitor?
Simplicity is admirable, but how does this improve customer satisfaction?

If you can define this in simple, but effective terms, you will have a highly effective value proposition.

Be prepared to combat stereotypes

Every sector has its own negative stereotype that can be harmful even before they get started.

Delivery services are known for not turning up when agreed upon, car salesmen are known for being notoriously pushy and fast food is known for being convenient but unhealthy. These are all examples of initial reactions, or cognitive biases, that most people have before using a product or service.

One of the most powerful kinds of value propositions are those that go against the grain of expectations.

If your particular sector is known for a negative practice, ensuring that your brand is above the stereotype can provide a concise and enticing positioning statement.

Value proposition canvas

A value proposition canvas is a focused way of structuring the main components of the value proposition in order to provide the most concise solutions for potential customers.

Value proposition canvas segments

The value proposition canvas is made up of two segments: the customer segment and the value proposition.

Inside each segment are three sections. The customer segment (shown below in red) has the specific needs of the customer, while the value proposition segment (shown in green) has the corresponding solutions.

value proposition canvas

The three sections of the customer segment (red) in the value proposition canvas are:

Customer jobs – what tasks can your customer accomplish by using your product or service?
Customer pains – what problems, risks or negative experiences can the customer eliminate when using your product or service?
Customer gains – what unique benefit does the customer gain when
using your product or service?

The value proposition map (green) on the other side of the canvas seeks to address these factors while presenting unique solutions and unexpected gains.

Gain creators – this is where you detail the unique improvements and benefits that your product or service offers to your customers
Pain relievers – this is the area to address elements that eliminate a current frustration or pain point (such as undesired costs, accessibility, or negative stereotypes)
Products and services – this is the spot to list all of your products and service

How to create a value proposition canvas

Let’s start with the customer segment.

The first thing to consider when creating a value proposition canvas is the customer. The customer is the central part of the entire premise. Be sure to speak carefully to your specific audience and avoid making generalizations.

Customer pains are generally more straightforward; therefore, pain relievers are often easier to define.

It’s tempting to focus solely on the functional factors, such as how to get from A to B. However, while the function is important, the emotional and social aspects also need deep consideration when developing your value proposition canvas.

Customer gains are a more complex element of the value proposition canvas as they can be slightly more difficult to define. In essence, these cannot be discovered without a substantial understanding of the uniqueness of the business itself. Not only will you list what the customer will gain from your product, but also what the customer will gain by choosing you over the competition.

Now let’s move on to the value proposition portion of the canvas.

Listing the potential jobs, or tasks, that customers want to accomplish with the help of your product or service is a good place to start.

Once completed, you will move on to the pain relievers and gains.

The pain relievers will specifically target the pains listed in the customer segment and should remain consistent. Meanwhile, the gain creators will show where your product or service adds value.

It’s important to remember that gains aren’t the opposite of pains. They are different factors that ultimately encourage consumers to adopt the product or service.

In the gains section, you will address the lesser-known desires of the customers – these are things that the customer themselves may not even be aware of. The lesser-known customer needs can be considered the “bonuses” that your brand offers potential customers.

Let’s go back to Uber as an example. Uber’s customers need quick, accessible and affordable transportation. These are the customer’s first thoughts. In addition, their customers also need safety during their rides. Uber offers insurance during trips to their drivers and passengers to cover them in the case of an accident. Furthermore, they can provide passengers with more comfort by including driver ratings from previous passengers. Customers may not immediately think of these factors, but they address very important human needs.

After development, each part of the right side of the canvas should be balanced with solutions and gains on the left. Ultimately, a value proposition canvas seeks perfect alignment.

Finding your fit in the market

Creating a value proposition canvas is a great tool to help understand your product, where you fit in the market, how you stand against competitors and the best way to market your brand successfully.

Drawing conclusions from this canvas and analyzing your value proposition can help you improve your strategies, messaging and overall product. A unique and profound value proposition is the core of a strong business.

You might also like...

See all

Article

2min read

How We Made AB Tasty’s Feature Experimentation SDK OpenFeature-Compatible — And Even Easier to Adopt

Julie Dumont

Apr 30, 2025

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Subscribe to
our Newsletter

Article

Mar 17, 2023

15min read

Prevent and Manage Technical Debt Using Feature Flags

AB Tasty

In modern software development, teams often have to prioritize speed and less than ideal solutions to put out products quickly to keep up with fast-changing consumer demands.

Unfortunately, taking such shortcuts could have dire consequences in the form of heavy costs or technical debt that could take a toll on your code quality and your whole software development and delivery processes if left unattended.

In this article, we’ll explore what technical debt is, the causes and different types of technical debt as well as how to manage it, largely through the use of feature flags.

What is technical debt?

The term “technical debt” was first coined by Ward Cunningham, one of the authors of the Agile Manifesto, in the early 1990s. Since then, the term has gained momentum and is a serious issue that many tech teams today still struggle to manage properly.

His reason for its name is that technical debt bears direct correlations with financial debt. Software development teams can take shortcuts to satisfy immediate business requirements, but the debt plus accrued interest will have to be paid at a later stage.

Technical debt is the consequence of action taken by software development teams to expedite the delivery of a software application or specific feature which later needs to be refactored or redeveloped.

Put simply, technical debt refers to the build up of technical issues during software development due to a number of causes which we’ll discuss in the next section.

If not attended to, technical debt can spiral out of control, resulting in the total breakdown of the software development and maintenance lifecycle.

Therefore, it is critical to ensure that DevOps and software development teams pay close attention to technical debt management and technical debt reduction methods.

Here are some warning signs to look out for:

Buggy, difficult to maintain code
Unstable production environments
Bug fixes introduce more bugs
Data inconsistency
Decreased development pace and bottlenecks

What causes technical debt?

We can deduce that technical debt comes mainly as a result of delivering a release quickly at the expense of “perfect” code.

In other words, it often comes as a consequence of ineffective and inadequate practices to build software for a short-term benefit in the interest of saving time.

That is one major cause but it’s also more complex than that as technical debt can be due to a number of other reasons.

Some causes behind technical debt include:

Time pressure: Teams today are under great pressure to deliver releases quicker than ever before to remain competitive and meet consumer demands fast.
Poor code: This could be due to a number of reasons including use of tools without proper documentation or training.
Insufficient software testing: Lack of QA support or automated testing means a lot of bugs could remain in the code undetected which gives rise to technical debt.
Outdated technology: Over time, many technologies become obsolete and are no longer supported and could become a source of debt.
Lack of skill: Teams can sometimes unknowingly incur debt because they lack the skills to write better code. For example, having junior developers working on building complex software beyond their skill and experience level is a sure way to accumulate debt fast.

Over time, all these factors could result in accumulation of debt that will need to be addressed. The real danger is not actually having the debt in the first place- as often that’s inevitable- but it’s allowing this debt to build up with no plan or strategy to pay it off in the future.

Types of technical debt

There are many ways to classify technical debt. One of the most popular ways comes from Martin Fowler’s technical debt quadrant.

The quadrant is based on the idea of not whether something should be considered debt per se but rather whether this debt can be considered prudent.

What does this mean exactly? Think of it as a way of answering the question of whether all technical debt is bad and the answer, according to the quadrant, would be “it depends.”

Martin Fowler’s technical debt quadrant seeks to categorize the types of technical debt according to intent and context.

Generally speaking, there are two overarching types of technical debt: intentional and unintentional (deliberate vs inadvertent).

Intentional technical debt occurs when software development teams choose to leave their code as it is, without refactoring or improving it, to reduce the time-to-market metrics. In other words, they choose to incur technical debt.

Unintentional technical debt, for its part, occurs when poor code is written and so the code quality will need to be improved over time.

Suffice to say, as soon as these technical debt-causing issues are highlighted, it is imperative to fix them as quickly as possible.

Let’s take a closer look at the 4 main types of technical debt, according to Martin Fowler:

Reckless/deliberate: Teams possess the knowledge to carry out the task but decide to go for a “quick and poor quality” solution to save time and for quick implementation.
Prudent/deliberate: Teams are aware of the debt they’re taking on but decide that the payoff for an earlier release exceeds the costs. However, in this scenario unlike the above, teams have a plan on how to deal with the repercussions of taking on this debt.
Reckless/inadvertent: This is arguably the least desired form of debt where teams don’t have enough experience and blindly implement a solution without applying best practices. As a result, they’re unconscious of the fact that technical debt is being accumulated. Thus, no real plan to address this debt can be formulated.
Prudent/inadvertent: This occurs when teams apply best practices during software development but still accumulate debt due to unexpected coding mistakes. Thus, this type of debt is unintentional. Teams have the necessary skill and knowledge to identify and pay off the debt but the experience serves as a learning opportunity for developers to optimize and improve the code for future projects.

When it comes down to it, deciding what to classify as technical debt is not always black or white. It requires putting things into context first. This is especially important when you think of the pressure on teams to put out products quickly to meet consumer and market demands.

This means that they will constantly have to deal with the dilemma between taking on technical debt or delaying a release. However, it’s more of a matter of how to deal with and manage this debt rather than avoiding it completely- which may not always be possible- to minimize negative impact as much as possible.

Types of technical debt to avoid

At this juncture, it is reasonable to conclude that teams should avoid technical debt. Additionally, it is imperative to minimize and eliminate tech debt, particularly reckless and deliberate code debt.

Over time, technical debt could become too expensive to fix the longer it remains unfixed as “interest” builds up the same way that financial debt accrues interest. Eventually, technical debt can lead to a code becoming harder to maintain as the foundation of the codebase deteriorates. This will ultimately result in lower-quality products with the company reputation taking a major hit.

Prudent tech debt is the partial exception to this rule. This form of code debt can benefit software development organizations as part of the reducing time-to-value methodology.

In other words, the advantages of delivering a product to market as soon as possible can outweigh the cost incurred by technical debt. However, it is critical to monitor the tech debt to ensure that its value does not spiral out of control, negating the benefits of the reduced time-to-value exercise.

How feature flags can help with technical debt

Feature flags can help reduce the technical debt accumulated during the development, testing, and deployment of a software application.

However, if feature flags are not monitored and maintained, they can increase the application’s technical debt.

Before we look at how feature flags reduce technical debt, let’s take a quick look at what a feature flag is:

“Feature toggles [feature flags] are among the most powerful methods to support continuous integration and continuous delivery (CI/CD)… [They] are a method for modifying features at runtime without modifying code.”

One of the most common sources of technical debt is the pressure to release a version of the software application.

The business demands that the software be deployed, and they don’t care how the developers make it happen. Feature flags are a valuable tool to help manage the “pressure-cooker” release environment.

There are several benefits to the use of feature flags as a software release and deployment aid, including:

The risk of deploying a bug-ridden application is substantially reduced. Developers can simply switch off features that are not yet complete or thoroughly tested.
By implementing a CI/CD methodology (continuous integration/continuous delivery), developers can often use feature flags to deploy new features without waiting for the next release to be deployed. In summary, this functionality reduces the time-to-value and increases customer satisfaction: A win-win for all.
Implementing feature flags is also a means to negotiate with management about which functionality to complete before specified deadline dates, increasing the flexibility to develop and test features thoroughly before deploying them.

In summary, feature flags help manage and reduce technical debt by helping software development teams manage the development/testing/deployment lifecycle more effectively.

Feature flags are useful for dark launching, a practice of releasing specific features to a subset of your user base to determine what the response is to a new feature or set of new features. As an aside, this is also known as a canary deployment or canary testing.

Testing in production is another form of dark launching. By utilizing this option, you can assess the application’s health, collect real-world app usage metrics, and validate that the software application delivers what your customers want.

Feature flags can also create technical debt. While they play a significant role in mitigating technical debt in all other areas of the software development lifecycle, implementing them is usually via a set of complex if-else statements.

Therefore, in practice, a feature flag is an if statement defining the path between at least two different code paths depending on one or more conditions.

The following simple scenario describes how to implement feature flags.

Let’s assume that an e-commerce site offers free shipping for all customers that spend more than a specified minimum amount at one time.

This code sample is an example of a feature flag. If the total amount paid is more than $50, then the shipping is free. Otherwise, the shipping amount is the amount spent multiplied by the rate (a percentage of the total amount).


def ShippingYN(amt, rate) :
  if amt > 50.0 :
    shipping = 0.0
  else :
    shipping = amount * rate

  return shipping

Best practices using feature flags to avoid technical debt

As with all aspects of software development and deployment, it is vital to observe the following feature flag best practices:

1. Feature flag management

As your organization matures in its use of feature flags as an integral part of the software development/testing/deployment lifecycle, it is vital to be mindful of the fact that some of the feature flags are short-term and should be removed; otherwise, they will add to the application’s complexity, resulting in more technical debt.

Consequently, it is imperative to have a plan in place to remove the flags before even setting them.

It is also possible, and a good idea, to track and measure different metrics for each feature flag, such as how long it has been active, its states (on/off), different configurations, and how many code iterations it has been through.

Once your feature flag has been through the required number of iterations to code and test a feature, this flag must be removed and the code merged into your code repository.

Note: Before removing a feature flag, it is a good idea to evaluate its function and purpose; otherwise, there is a risk, albeit slight, that the flag might still be needed and is erroneously removed.

A vital part of the feature flag management process is to define and implement temporary and permanent flags.

1.1 Temporary feature flags

As highlighted above, if a feature is designed to be rolled out to every application user or you are using the feature as a short-term experiment, it is critical to attach a monitoring ticket to this flag to make sure it is removed once the feature has been deployed or the experiment is concluded.

Examples of these temporary flags which can last weeks, months, or even quarters, include:

Performance experiments: A performance experiment is similar to A/B testing, where two versions of the feature are deployed with the idea of determining which one performs better. A/B testing employs the same construct in that it deploys two versions of an element to the application’s user base to select which one users prefer.

Painted-door experiments: These experiments are only used in the early phases of the software development lifecycle and are User-Interface mock-ups to determine any customer interest. Once the consumer interest has been determined, these flags can be removed.

Large-scale code refactoring: It is a good idea to deploy code refactoring changes behind a feature flag until you are positive that the functionality has not been changed or broken. Once the refactoring exercise is complete, you can remove these feature flags.

1.2 Permanent feature flags

Permanent feature flags are used to implement different features or functionality for different groups of users.

As a result, it is reasonable to assume that these flags will remain in the software application indefinitely or at least for a very long time.

Therefore, it is vital to ensure that they are monitored, documented, and reviewed regularly. As with the temporary flags, there are several different types, including:

Permission flags: These feature flags are helpful when your product has different permission levels, such as the ability to create journal entries in an online financial general ledger or whether users can view a list of these entries. A second use case for these flags is your SaaS application has different subscription models like Basic, Professional, and Enterprise.
Promotional flags: These flags help implement regular promotions. For instance, let’s assume your e-commerce store offers a Mother’s Day promotion every year where specific products bought include the shipping costs.
Configuration-based software flags: Any software driven by config files will benefit from using feature flags to implement the different possible configurations. A typical use case for config flags is the layout of the User Interface.
Operational flags: These feature flags help manage a distributed cloud-based application. For example, additional compute engines can be spun up when the workload reaches a specific level.

2. Use a central code repository

Feature flags or toggles are most commonly stored in config files.

Another option is to keep them in a database table. However, let’s look at how to manage these config files. Large systems can have many if not hundreds of feature flag settings. Apart from using a database table, the only way to manage these settings is to store them in config files.

The best way to maintain the config files is to upload these files to a feature flag library in a central code repository like Git.

Not only is Git good for keeping control of these files, but it is also a valuable version control system. Developers can use it to create feature branches of config files used during the software development process without negatively affecting the production version of these files.

Once the config files have been updated and tested, they can be merged back into the Git master branch using a merge request.

3. Adhere to naming conventions

It is absolutely critical to give your flags intuitive, easy-to-understand names, especially for long-term flags, although it is a good idea to include short-term flags in this best practice.

Naming the feature flags, flag 1, flag 2… flag 100 will not help people who have to work with these flags in the future.

A good example of wisely named feature flags can be found in the scenario highlighted above.

It is reasonable to assume that the flag, AdvancedSearchYN, is one of hundreds of flags used in our eCommerce application. Even if they are the only two flags used, it is still advisable to give them intuitive, related names.

For more details on the best way to manage feature flags to keep technical debt at bay, download our feature flag best practices e-book.

4. Use a feature management system

Using a dedicated feature flagging system is a great way to manage flags in your code so you don’t find yourself with piles of technical debt from unused or stale flags.

AB Tasty’s server-side feature enables you to remotely manage feature flags and take control over how, when and to who features are deployed to mitigate risk while optimizing user experience.

To help with technical debt management, AB Tasty provides dedicated features to keep control over your feature flags. Two of them are especially useful in this regard:

The Flag Tracking dashboard lists all flags setup in the platform, with their current values (ex: on/off) and campaigns that reference them. This way, you can easily keep track of every single flag purpose (ex: flag 1 is used in progressive rollout campaign X, while flag 2 is used in feature experiment Y). When you manage hundreds of flags, it turns out to be a real time saver.
The Code Analyzer is an advanced Git integration to see where your flags are used in your repository. In conjunction with the flag tracking dashboard, you can quickly find flags in your code that are not referenced in any campaigns. It also deeply integrates with your current CI/CD pipeline. As a CLI and a Docker image, it analyzes your codebase and detects the usage of flags everytime code is pushed to a specific branch or tag. This way, your flag dashboard is always in sync with your codebase. On one hand, you can safely remove flags if they are not referenced in campaigns, and on the other hand you make sure that flags your campaign is relying on are indeed available in your code. View code on Github.

Feature flag references in github/gitlab codebase — The Flag Tracking dashboard with flag references to the codebase

Try it for free!

Final thoughts

As described throughout this article, feature flags in DevOps and software development play a fundamental role in managing and reducing technical debt.

Consequently, it is vital to implement a feature flags framework as a foundational part of the software development lifecycle.

Cobbling it on afterward can increase the risk of incurring more technical debt, especially once the system grows in scale. Thus, these feature flags must be carefully maintained and monitored to ensure that they don’t amass additional technical debt.

Finally, it is essential to be mindful that, while technical debt is primarily seen as a negative, there are instances, as described by Martin Fowler’s technical debt quadrant, where incurring prudent and deliberate tech debt can be beneficial.

It is also worth noting that both Agile and Scrum use the concept of technical debt in a positive way to reduce the time-to-value of a new application or feature release, driving sustainable growth through customer satisfaction.

You might also like...

See all

Article

2min read

How We Made AB Tasty’s Feature Experimentation SDK OpenFeature-Compatible — And Even Easier to Adopt

Julie Dumont

Apr 30, 2025

Article

7min read

Understanding the Impact of Tariffs on E-Commerce Pricing Elasticity

Edoardo Aliprandi

Apr 15, 2025

Article

3min read

Tag V4: Elevating your Experience Building with an Improved Modification Engine

Julie Dumont

Feb 25, 2025

Subscribe to
our Newsletter

Article

Mar 16, 2023

4min read

AB Tasty User Club: Empowering Customers Through Better Feedback and Engagement

AB Tasty

At AB Tasty, we have always put client feedback at the heart of our product roadmap. Listening to our clients’ needs and helping them achieve their goals is a top priority for us. We just don’t say it, we do it:

🎁 55 new features brought to users in 2022
📣 10 market releases per year
🤝 1545 feedback requests processed

To go one step further, we have decided to launch our User Club! 🎉

The AB Tasty User Club is a new opportunity for you to share your feedback, experiences, and needs with us. Being part of the Club means you’ll have exclusive access to:

Our new features
A way to interact directly with our Product Managers and Designers
Our product related events
A real community where you can share your usage and hear best practices from other users

A successful launchpad for our User Club

As a first step, we organized our first User Games of the year in January 2023 in Paris, France on the theme of data, monitoring and performance analysis in CRO activities.

This event was a great success for our Product teams and for the 5 customers who attended to discuss their data understanding and analysis needs. We welcomed participants from different industries, all interested in data analysis techniques and how to use them to improve products and services. We also invited experts in the field of data analysis to share their experience and knowledge.

AB Tasty User Club

Baptiste Deroche, Product Designer @AB Tasty:

“This event was the perfect opportunity to validate and challenge assumptions we have about the product. We learned a lot from our customers that day, and it’s a really good start to getting closer and closer to our end users.”

Stéphanie Duchemin, Product Design Team Leader @AB Tasty:
“It was a pleasure to meet our users again in a real session and not remotely, and I think that the pleasure was shared. This reinforces my conviction that feedback is not the same in a face-to-face session as in a remote one. Through our discussions we learned and discovered some pain points that were not necessarily related to the initial topic and that will feed our roadmap for at least 6 months!”

Mariza Baxevani, Web Conversion Specialist @Prestashop:

“It was a collaborative and enriching moment where each participant presented their feedback, their experimentation process and their vision of the tool. It’s really reassuring to know that AB Tasty values its users’ feedback. I will gladly participate in this type of event again!”

The AB Tasty User Club was created to offer our customers a space to discuss and share their opinions and suggestions. We have received a lot of positive feedback from participants and this gives us even more motivation to create other similar projects and events where you will be at the forefront.

Stay tuned for the upcoming events or announcements! If you are not part of the Club yet, do not hesitate to talk about it with your dedicated Customer Success Manager!