Tag: EN

Article

Dec 21, 2022

10min read

The Ethical Use of First-Party Data for Personalization

AB Tasty

With the end of third-party cookies in sight, first-party data has moved to the forefront of digital marketing.

First-party data is a powerful tool for personalizing your customers’ buying journey. It’s generally more reliable and offers deeper customer insights than third-party data, helping you gain that competitive edge. But these benefits also bring responsibility. It’s essential from both a compliance and customer experience perspective that you practice ethical data collection when it comes to first-party data.

In this article, we take a closer look at first-party data—what it is, how you can collect and use it ethically and the benefits first-party data offers both your customers and your business.

What is first-party data?

First-party data is information about your customers that you collect directly from them via channels you own.

Potential sources of first-party data include your website, social media account, subscriptions, online chat or call center transcripts or customer surveys. Importantly, the first-party data you collect is yours and you have complete control over its usage.

Examples of first-party data include a customer’s

name, location and email address
survey responses
purchase history
loyalty status
search history
email open, click or bounce rates
interest profile
website or app navigational behavior, including the page they visit and the time they spend on them
interactions with paid ads
feedback

As it comes straight from the customer, first-party data provides you with deep and accurate insights into your audience, their buying behavior and preferences.

These insights are essential for guiding the development of digital marketing strategies that prioritize the human experience, such as personalization. They can also help you create customer personas to help connect with new audiences which may inform key business decisions, including new products or services.

How to collect first-party data

Customers may voluntarily provide first-party data. For example, customers submit their email addresses when signing up for a newsletter, offer their responses when completing a survey or leave comments on a social media post. This is often referred to as declarative data—personal information about your customers that comes from them.

Alternatively, first-party data can be collected via tracking pixels or first-party cookies that record customers’ interactions with your site. This produces behavioral data about your customers.

First-party data is typically stored on a Customer Data Platform (CDP) or Customer Relationship Management (CRM) Platform. From this, you can build a database of information that you can later use to generate customer personas and personalize your marketing efforts.

What is third-party data?

Third-party data removes the direct relationship between your business and your customers during the data collection process. While first-party data comes straight from your customers, third-party data is collected by a separate entity that has no connection to your audience or your business.

Unlike first-party data which is free to collect, third-party data is typically aggregated from various sources and then sold to businesses to use for marketing purposes.

From a marketing perspective, third-party data is further removed and therefore offers less accurate customer insights. You don’t know the source of third-party data and it likely comes from sources that have not used or don’t know your business, limiting its utility.

For many years, marketers relied on third-party cookies to provide the data needed to develop digital marketing strategies and campaigns. But over time, concerns around the ethics of third-party data collection grew, especially in relation to data privacy and users’ lack of control over their data. As a result, most of the major search engines have banned—or will soon ban, in the case of Google Chrome—the use of third-party cookies.

Is first-party data ethical?

First-party data is ethical if it’s collected, stored and used according to data privacy laws, regulations and best practices that require responsible and transparent data handling.

The move away from third-party cookies highlights how first-party data is preferable when it comes to ethical considerations. With full control over the data you collect, you can ensure your first-party data strategy protects the data privacy rights of your customers. You can clearly explain to your customers how you handle their data so they can decide whether they agree to it when using your site or service.

Unfortunately, unethical first-party data collection can and does happen. Businesses that collect data from their customers without informed consent or who use the data in a way the customer didn’t agree to—such as selling it to a third party—violate their data privacy. Not only does this carry potential legal consequences, but it also significantly undermines the relationship of trust between a business and its customers.

How do you collect first-party data ethically?

The first step towards ethical data handling is compliance. There is a range of data privacy laws protecting customer rights and placing obligations on businesses in terms of how they collect, store and use personal data, including first-party data.

Confirming which laws apply to your business and developing an understanding of your legal obligations under them is not only essential for compliance, but it also informs your data architecture structure. The application of data privacy laws depends on your business or activities meeting certain criteria. It’s worth noting that some data privacy laws apply based on where your customer is located, not your business.

Data privacy legislation in Europe

European customers’ data privacy is protected by the General Data Protection Regulation (GDPR). The GDPR requires businesses to demonstrate ethics in data collection and use.

This often means customers must provide informed consent, or opt-in, to their data being collected and used. Businesses must also keep records of this consent. Customers can withdraw their consent at any time and request their data be deleted in certain cases. You must implement reasonable security measures to ensure data is stored securely, according to the level of risk. One option is to use air-gap backups to protect data from cyber threats by isolating it from the network. In certain circumstances, you also need to nominate a data protection officer.

Data privacy legislation in the UK

If you have UK-based customers, you need to comply with the provisions of the UK General Data Protection Regulation (UK GDPR) and the Data Protection Act 2018. These include providing a lawful basis for collecting personal data, such as consumer consent via a positive opt-in.

Consumers have the right to request the use of their data be restricted or their data erased, in certain circumstances. Relevant to first-party data, consumers can object to their data being used for profiling, including for direct marketing purposes.

Data privacy legislation in the US

The US doesn’t have a federal data privacy law. Instead, an increasing number of states have introduced their own. The first state to do so was California.

Under the California Consumer Privacy Act (CCPA)*, you can only collect customer data by informed consent—customers need to know how data, including first-party data, is collected and used. Customers also have the right to opt-out of the sale of their personal data and to request their data be deleted. If a data breach occurs where you have failed to use reasonable security measures to store the data, customers have a right of action.

2023 looks to be a big year for the data privacy landscape in America. In Virginia, the Consumer Data Protection Act (VCDPA) is due to commence on January 1. The VCDPA includes a provision for customers to opt-out of data collection for profiling or targeted advertising processes. Colorado, Connecticut and Utah have introduced similar laws, also ready to commence next year.

Beyond compliance

As you can see, some general principles emerge across the different pieces of data privacy legislation:

Customer consent — customers should consent to the collection and use of their data
Transparency — you should explain to customers what data you collect, how you collect it and what you do with it, typically via a privacy policy or statement
Control — customers should be able to control the use of their data, including requesting its deletion.

From a consumer perspective, compliance is the bare minimum. While the design of your data architecture structure should be guided by the above principles and comply with any relevant data privacy laws, you can also take extra steps to demonstrate your business’s commitment to ethical data handling. This may include appointing a data protection officer to oversee compliance and provide a point of contact for complaints or providing your employees with training, even where it isn’t required by law.

How to use first-party data

In a crowded online marketplace, it’s hard to make yourself heard over the noise. Arming yourself with accurate and reliable first-party data, however, helps you stand out from the crowd and communicate your message to both current and potential customers.

Firstly, you can use the first-party data you collect to create an exceptional customer journey through personas—fictional representations of your customers’ broad wants and needs. Building a series of personas can help you tailor your product or service and business practices to better serve your general customer base.

First-party data is also a crucial ingredient for more specific 1:1 personalization. With it, you can craft a unique user experience for your customers by delivering individual recommendations, messages, ads, content and offers to improve their purchasing journey.

In addition to serving a marketing purpose, first-party data is also essential for retargeting customers, for example, by sending abandoned cart emails. It can also help you identify and address gaps in your customers’ buying experience or your current offerings.

Want to get started with 1:1 personalization or personal recommendations?

AB Tasty and Epoq is the complete platform for experimentation, content personalization, and AI-powered recommendations equipped with the tools you need to create a richer digital experience for your customers — fast. With embedded AI and automation, this platform can help you achieve omnichannel personalization and revolutionize your brand and product experiences.

Benefits of first-party data

Personalization

First-party data provides deeper insights than second or third-party data, allowing you to incorporate a higher degree of personalization in your marketing. In turn, this improves the buying experience for your customers, gaining their loyalty.

Reduces costs

Engaging a third party to aggregate data costs money. First-party data, on the other hand, doesn’t cost you anything to collect.

Increases accuracy

Collecting data from your specific customer base and their interactions with your company produces tailored insights, rather than generic information. First-party data comes directly from the source, increasing its reliability.

Gives you control over data

You own first-party data collected from your customers. This puts you in full control of how it is collected, stored and used.

Transparency

As you have full control over how you collect and use first-party data, you can clearly explain this to your customers to obtain their informed consent. This transparency builds trust and loyalty with your customer base.

Strengthens customer relationships

In a recent Ipsos poll, 84% of Americans report being at least somewhat concerned about the safety of the personal data they provide on the internet. At the same time, Salesforce found that 61% of consumers are comfortable with businesses using their personal data in a beneficial and transparent way. First-party data builds better customer relationships by balancing customers’ desire for data privacy with their preference for personalized advertising.

Compliance with regional privacy laws

Most countries are strengthening their legislative framework around data privacy and prioritizing users’ rights. With first-party data, you can design your data architecture structure to ensure it complies with any relevant laws.

Ethical first-party data handling benefits both you and your customers

First-party data is the key to accurate and sharp customer insights that help you shape effective, targeted marketing strategies. But with the demand for ethical data collection at an all-time high, it’s important you treat your customers’ first-party data with care.

First-party data should be collected responsibly and transparently, with the customer’s fully informed consent. Your first-party data strategy also needs to comply with any relevant data privacy laws, regulations and best practices. This approach achieves a happy medium between addressing customers’ data privacy concerns with their desire for personalization during the purchasing journey. It also helps you optimize your customer’s experience with your business and, in turn, your profits.

Interested in learning more about how you can use first-party data to benefit your business? Check out our customer-centric data series for more insights from the experts.

*Amendments to the CCPA are due to be introduced in 2023, via the California Privacy Rights Act. Many of the related regulations are still being updated.

You might also like...

See all

Article

4min read

Beauty E-Commerce Gets a Glow-Up: Insights from Our Cosmetics Consumer Report

Kate Feng

Aug 21, 2025

Article

7min read

Winning Big During the E-Commerce Holiday Season: Strategies and Insights

Brian Heberer

Aug 18, 2025

Article

3min read

Unlock Any Audience Source with AB Tasty’s Universal Connector

Gaetan Postorivo

Aug 1, 2025

Subscribe to
our Newsletter

Article

Dec 9, 2022

10min read

Measure your DevOps Performance: DORA Metrics

Rowan Haddad

Nowadays, as software development processes become more decentralized and as the number of teams working on different projects (in different places too) increases, it becomes that much harder to set and track metrics to measure performance across these teams.

And yet data is now more important than ever. Data is a company’s most valuable asset in order to measure how efficiently teams are performing over time to deliver the best products and user experience to customers.

This is especially relevant for DevOps teams where there’s a need for a clear framework to measure their performance.

This is where DORA metrics come in.

What are DORA metrics?

DORA metrics provide a standard framework to help leaders who are implementing a DevOps methodology in their organization to measure the performance of their teams.

This framework was the result of a six-year research program conducted by Google Cloud’s DevOps Research and Assessment (DORA) team after analyzing survey responses from over 32,000 professionals worldwide. Their goal was to determine the most effective ways to develop and deliver software.

Through the use of behavioral science, the research identified four key metrics that would indicate the performance of a software development team.

With these metrics, teams can measure their software delivery performance, monitor it over a period of time and be able to easily identify areas of improvement to optimize performance. In that sense, they shed light on the capabilities that drive high performance in technology delivery.

Therefore, DORA metrics are especially relevant for DevOps teams as they provide them with concrete data to measure performance and improve the effectiveness of their DevOps operations. It also allows them to assess whether they are building and delivering software that meets customer requirements as well as gain insights on how to improve and provide more value for customers.

The four DORA metrics

In this section, we will list the four main metrics that the DORA team identified for DevOps teams to measure their performance.

The following chart shows from the 2022 State of DevOps report, updated each year, shows the ranges of each metric according to the different categories of performers:

The four key metrics used are:

Deployment frequency

Deployment frequency measures velocity. In this case, the goal is to measure how often an organization successfully deploys code to production or releases it to end users.

This is an important metric particularly for DevOps teams whose ultimate goal is to release software quickly and frequently. It helps teams to measure their productivity and agility as well as uncover issues and bottlenecks in their workflow that may be slowing things down.

High performing teams deploy on-demand, multiple times a day. Thus, this metric stresses the importance of continuous development and deployment, which is one of the principles of a DevOps methodology.

Each organization will need to consider what constitutes a “successful” deployment for its teams such as taking into account what level of traffic is sufficient to represent a successful deployment.

How to improve this metric:

To enhance this metric, it’s usually best to ship code in small batches on a frequent basis. This will allow you to reduce risk of deploying bugs and increase speed of delivery. Implementing an automated CI/CD pipeline will also enable you to increase deployment speed.

Lead time for changes

Lead time for changes is the amount of time it takes a commit to get into production. Therefore, this metric also seeks to measure velocity and gives an indication of a team’s cycle time. The lower the lead time for changes, the more efficient the team is at deploying code.

This metric requires looking at two pieces of data: when the commit happened and when it was deployed. The goal is to keep track of the time development starts until the committed code is finished and deployed to uncover any inefficiencies in a team’s processes. The average time can then be used to analyze overall performance.

In other words, the purpose of this metric is to give an indication of the waiting time between the initial stages of implementing the change and deployment. A high lead time may suggest inefficiencies in the CI/CD pipeline and not enough automation, especially if every change has to go through manual testing instead which significantly slows things down.

How to improve this metric:

Again, here it’s best to work with smaller changes. This allows for faster feedback so developers can immediately fix any issues. Teams should also eliminate bottlenecks and integrate automated testing at every stage of the CI/CD pipeline to detect issues early on.

Feature flags are also a great tool to lower lead time as any unfinished changes can be hidden behind a flag while other changes can be deployed.

Change failure rate

This represents the number of deployments causing a failure in production. In other words, it measures any changes to code that resulted in incidents, rollbacks or any other failures. This depends on the number of deployments attempted and how many of those resulted in failures in production.

As a result, this metric is a measure of the stability and quality while the previous two focus mainly on speed of software delivery.

This metric requires the number of deployments that resulted in failures divided by the total number of deployments. The percentage calculated will give insight into how much time is dedicated to fixing errors as opposed to delivering new code.

The lower the rate the better. High performing teams have a change failure rate of 0-15%.

Consequently, teams with a low change failure rate is a sign that these teams have an efficient deployment process in place, which can be mainly achieved through automating every step of the process to avoid common manual errors.

It’s important to note, however, that this metric can be hard to quantify as the definition of failure can vary widely. Therefore, it’s best for each organization to set goals for its teams according to their unique business objectives.

How to improve this metric:

Automation is crucial to also help improve this metric. Automated tests can evaluate code at every stage in its development. This way, issues are caught and fixed early on so they’re less likely to make it to production. Creating critical feedback loops are necessary to get a low change failure rate to prevent incidents like this from happening again in the future.

Time to restore service

Also referred to as “mean time to recovery MTTR”, this indicates how long it takes for an organization to recover from a failure in production that impacts user experience.

This metric, like change failure rate, is meant to determine the stability of a system or application when unplanned outages occur. Thus, information about when the incident occurred and when it was resolved then deployed will be needed to measure the time to restore service.

Therefore, the “time to restore service” metric is important as it encourages teams to build more stable systems and create action plans to be able to respond immediately to any failures. High performing teams will resort to deploying in small batches to reduce risk while increasing speed of delivery.

This is particularly applicable to DevOps teams as they place high emphasis on the idea of continuous monitoring, which will in turn help them to improve their performance when it comes to this metric.

How to improve this metric:

Consider using feature flags. Feature flags act as switches enabling you to turn a change on or off in production. This means that if any issue occurs, you can toggle the switch off if something goes wrong with a change in production with minimal disruption while it’s being resolved. This will then help reduce your MTTR.

The DORA metrics can then be compiled into a dashboard. To do so, DORA created the Four Keys dashboard template to generate data based on the metrics and visualize the results. See example below of this dashboard:

The dashboard gives a higher-level view for senior stakeholders of their organization’s DORA metrics to understand how their teams are performing and what corrections can be done to remedy any problems.

Why are DORA metrics important?

As we’ve already mentioned, DORA metrics are a great way to keep track of the performance of DevOps teams and identify areas of improvement.

It helps organizations assess their delivery process and encourage teams to streamline their processes and increase the speed of delivery while maintaining quality.

As a result, the main benefits of these metrics are:

More effective decision-making– with the data acquired from these metrics, teams know what aspects to focus on that need improvement. Teams can easily detect issues and bottlenecks within the software development process and devise an action plan to address them. Decisions will be based on data rather than opinions or gut feelings which may be misleading.

Better value- DORA metrics will give teams an indication whether they’re delivering value to customers by evaluating the efficiency of your value stream and finding areas to improve within your delivery process to build higher quality products.

Continuous improvement- this is particularly important as it is one of the main pillars of a DevOps methodology. Using DORA metrics, the team gets insight on their performance and set goals to improve the quality and delivery of software.

Challenges of DORA metrics

DORA metrics have a lot of advantages, but they do come with their own challenges as well.

One of the main challenges when faced with these metrics is that they will vary across organizations and teams as, often, they have different definitions and processes in place. In other words, no products or teams are the same and may operate at their own level of complexity. As a result, it’s important to put this data into context before making decisions.

DORA metrics give a good overall picture of how teams are performing in certain categories. This means that it’s important to have a valid way to keep track of the data but don’t rely solely on them.

Teams may be facing issues beyond what is accounted for in these metrics. DORA metrics are focused mainly on the bottom line instead of the inputs and processes that lead to the outputs being measured. Sometimes, there’s more to the story than what DORA metrics measure so tread carefully.

Ultimately, enhancing performance will be unique to each organization. Work on shifting your attention to your team and goals to give context to the story all these metrics are telling. Focus on building the right culture for your team and providing them with the tools they need to enhance performance. This, in turn, will help them deliver business value faster.

DORA metrics: The key to unlocking more value

DORA metrics are a great starting point, especially to help teams make informed decisions about what can be improved and the steps to take to achieve that.

They give a good indication of a team’s progress along their DevOps journey and encourage the implementation of the key principles of DevOps including shipping in small batches more frequently.

In particular, it enables them to assess and analyze the efficiency of their development and delivery processes by offering a framework for measuring performance across two important variables in DevOps: speed (deployment frequency & lead time for changes) and stability (change failure rate & time to restore service).

Teams will then be able to create more value for their customers faster. Above all, DORA metrics are a way for teams to shift their focus to maximizing velocity and stability.

It’s important to note that tracking these metrics should be in line with your organizational goals and customers’ needs to give context to these metrics, make sense of them and improve them.

Feature flags are also another effective way to improve performance across these metrics. They will allow you to ship new changes in small batches and hide any that are not yet ready to speed up deployment while reducing risk of big bang releases making problems easier to detect and resolve.

Get a demo of AB Tasty to unlock the value feature flags can bring to your teams today.

You might also like...

See all

Article

4min read

Beauty E-Commerce Gets a Glow-Up: Insights from Our Cosmetics Consumer Report

Kate Feng

Aug 21, 2025

Article

7min read

Winning Big During the E-Commerce Holiday Season: Strategies and Insights

Brian Heberer

Aug 18, 2025

Article

3min read

Unlock Any Audience Source with AB Tasty’s Universal Connector

Gaetan Postorivo

Aug 1, 2025

Subscribe to
our Newsletter

Article

Dec 7, 2022

7min read

Understanding User Behavior Through Data

John Hughes

As part of our customer-centric data series, we are speaking with AB Tasty partners about how brands can use data to better profile customers, understand their needs, and forge valuable emotional connections with them, as well as measure overall digital impact and build a data-based, customer-centric vision.

You can catch the beginning of the series here. For our 8th installment, we spoke with Helen Wilmot, UX Director at Dentsu International. Dentsu is one the largest marketing and digital companies in the world and provides communication, marketing and digital strategies across a range of disciplines. Helen leads the UX strategy and research for Dentsu.

Why customers come to Dentsu

Dentsu is an integrated global company. People are often looking for the different channels they operate in, as well as having several disciplines collaborating under one roof to serve a client. More than that, Dentsu is a multi-national company with a network of global expertise focused on how to provide value to clients so that they actually see that they are linked to something that’s larger. Specific to the UX and optimization team, they are tightly integrated with other channels such as brand and digital strategists, SEO, and commercial. All of those aspects are taken on board when they are looking at optimization.

That means growth is one of the major factors at play: “We are firmly focused on being customer-centric and providing value for the users,” says Helen. “We also live in the real world and we want to tie that value to real growth. We know that being customer-centric drives growth, study after study after study shows that, but it’s important that we can show that ourselves in our own behaviors.”

Understanding user behavior

Helen stresses that there is no single way to understand user behavior and as such it will often depend on what is appropriate for the research or business problem at hand. Dentsu combines many techniques when looking to understand customer behavior, often a balance of qualitative and quantitative data. In many cases they are looking for a breadth of data to make it accurate – interviews, usability testing, AI, eye tracking, data insights, and card sorting all play a part.

The techniques employed depend on the customer and the business model. Consistently, however, they are led by data. Researchers perform usability tests, both at a distance via remote studies and in-person moderated use testing. Having users try something out in their own environment using their own devices also helps Dentsu asses more natural behavior. They sometimes employ ethnographic techniques, which are borrowed from anthropology, as researchers are embedded in users’ real experiences. Looking at interaction with a topic or task in real-time, this research is more generative and exploratory but can help uncover larger issues. New technology in behavioral science and neuroscience can look at emotions, implicit response testing and eye behavior.

What customers say they want and how they actually behave

Specifically, when Helen’s team conducts interviews they look at how users behave as well as what they say.

“User testing should be about observing,” shares Helen. “If the user said they found the task easy, but they clearly didn’t, it’s a sign to use our critical thinking to evaluate that feedback.”

So Dentsu always dig deeper to back those statements up with data. They have a rigorous approach to their research, including the use of a laddering technique used to get to the root cause of people’s thoughts and experiences. “It’s all working within the reality of how we know people’s minds work. We know that people are terrible at remembering exactly how they felt and terrible at predicting our own behavior. So that’s not to say that experiences don’t matter but as researchers, our job is to work within the reality of how human psychology works too,” says Helen.

The importance of testing and experimentation

Another way Dentsu works to validate user feedback is with a reliance on A/B testing. In the past, they have had users report certain things that motivate them, but the A/B tests do not back up that information. For Dentsu, testing out a hypothesis is a crucial part of their optimization process and it is unthinkable that they would go ahead with new ideas without testing them first.

“I think it’s a bit of an act of hubris If you don’t go ahead and test,” stresses Helen. “The risk just absolutely shoots up, and you can’t de-risk a solution without testing it. Even if a solution is successful and is fit for purpose, there are always iterative changes that you can make. That is the beauty of a testing and experimentation mindset. You are never finished, and we never think that what we do is perfect.”

Testing allows Dentsu to move at speed. Some tests don’t always work, but Helen points out that losing tests also brings a wealth of information for other ideas. Knowing something doesn’t work for your users informs you for your next test and when you are working with future customers, it can point you in the right direction as to what techniques are currently effective for messaging and psychological persuasion.

The KPI’s and metrics within user behavior

For UX, one if the main KPIs is NPS (Net Performance Score), just as it was for Realise. In usability tests, Dentsu will look at the number of errors someone makes. If they are testing a certain structure or tree testing, they will look at success rate and directness. NPS compliments all of these because it has numerous studies behind it associated with growth. There is also a holistic view to take when it comes to user behavior metrics: what areas within the business are being affected by friction on the website? It could be customer support, payment or delivery, but they are all vital to the user experience.

A good user journey

A good user journey will always provide revenue. The two are intrinsically linked for Dentsu. If you get it right, people become more engaged in the brand, more engaged with what you are offering and it makes them more likely to do what you want. Helen wants to go further, though. Usability and revenue are vital aspects, but they are only the beginning.

Helen explains, “I think usability is the absolute baseline that we should aim for, and we really should be focused on delighting customers while creating emotionally resonant experiences. There is strong data on the link between emotion, memory and brand perception and by creating these rich, emotionally resonant experiences we can boost lifetime value.

It seems that in the current climate, people are placing an emphasis on the bottom line and conversion, but, as Helen shares, the original thinkers on UX such as Donald Norman were looking to delight to provide insight into optimization. Brand and user experience are inevitably related and users who have an emotional connection with a brand will give you more and more chances to be present in their lives.

“It’s important to view delight as a bit of a North Star. If you make your users happy, you’ll make shareholders happy as well.”

You can find out more about our Customer Centric led by looking at our previous installment on how to solve real user problems with a CRO strategy.

You might also like...

See all

Article

4min read

Beauty E-Commerce Gets a Glow-Up: Insights from Our Cosmetics Consumer Report

Kate Feng

Aug 21, 2025

Article

7min read

Winning Big During the E-Commerce Holiday Season: Strategies and Insights

Brian Heberer

Aug 18, 2025

Article

3min read

Unlock Any Audience Source with AB Tasty’s Universal Connector

Gaetan Postorivo

Aug 1, 2025

Subscribe to
our Newsletter

Article

Dec 6, 2022

5min read

Test More, Risk Less: A/B Testing as a Risk Mitigation Tool

Rowan Haddad

Imagine this: You see a continual drop in conversions on your e-commerce website. In particular, customers are abandoning their carts and not completing their purchases.

Based on previous experience, you decide to take action by switching up a few elements on the checkout page.

But it’s still not getting you the results you’ve hoped for; instead, you see your conversions going even further down. What went wrong?

How experiments can come to the rescue

There’s always a certain degree of risk in business and any wrong move can be a potential loss in profit. How can you test your ideas without breaking your website and negatively impacting your business?

Thanks to A/B, A/B/C and multivariate testing, you can manage and even reduce that risk before making a big decision that could hurt your bottom line.

Before we go any further, let’s look at a simple definition of A/B testing.

A/B testing is a marketing technique that involves comparing two variations or versions of a web page or application, randomly presented to users, to see which performs better by evaluating how they impact your KPIs.

The results of such tests will help you assess the risk of your business decision as it presents an opportunity to gather feedback from the people with the most valuable opinions — your customers. Otherwise, you’d make the decision based on personal opinions rather than customer preferences, eventually leading you down the wrong path.

A/B testing helps steer you in the right direction by enabling you to test and learn quickly if something works or not before embedding it into the back-end or permanent coding.

If you don’t take steps to manage risks, you’d never be able to tell if your new ideas will resonate with your customers or if they’re worth the investment. However, when you run A/B tests, you can minimize the risk of any drastic business impact.

For example, you can make a pretty website by changing colors but if this change drives down conversions then you’ll have a nice looking but poor performing website. Thus, A/B testing gives you the green light to go through with an idea by monitoring how it’s affecting your KPIs.

Experimentation is the most efficient way to eliminate rolling out a bad idea for your website or prove the value of a change before investing time and resources. It’s a golden opportunity to learn what really drives conversions so you can use the data extracted to fully commit to any changes in UX.

Even if an experiment doesn’t turn out the way you planned, you can still use it as a learning experience on what your customers don’t want so that you stay on the right track.

To get the most out of your A/B tests, you need to leverage both quantitative and qualitative data when it comes to effective decision-making. In other words, running these experiments is important, but it’s the quick steps you take after based on the results that will make all the difference.

AB Tasty is a great example of an A/B testing tool that allows you to quickly set up tests and gather results to help you mitigate risk with ease. With low code implementation of front-end or UX changes on your web pages, you can gather insights via an ROI dashboard, determine which route will increase your revenue, and achieve the perfect digital experience for your customers.

In summary, always make sure you test new features or changes prior to release to make sure they take your business metrics in the right direction. Even if it seems like it’s a very minor change, it could still have a significant impact on your conversions and revenue.

Steer your A/B test in the right direction

While you’re running an experimentation campaign, there’s also another layer of risk when randomly allocating users to your variations as so often happens to be the case during an A/B test. What if there’s a variation that you notice is performing poorly? How can you quickly turn this off before more traffic is exposed to the variation?

Luckily, there’s a way to further minimize risk during an A/B test and that’s through dynamic allocation, a capability offered by advanced A/B testing solutions such as AB Tasty.

Dynamic allocation seeks to limit loss due to the lowest performing variations so that fewer visitors are sent to the “bad” variations to maximize outcome, as can be seen in the image below.

For example, if you run an A/B test with two variations with the goal to increase conversions on the checkout page and variation B is performing well and has a high conversion rate, then the traffic to that variation is adjusted (and increased) accordingly.

One of the advantages of dynamic allocation is risk mitigation. This will enable you to confidently and safely test new elements. If they don’t work out as predicted, then the traffic allocated will be gradually reduced so fewer users will access it.

Make risk-free and data-driven decisions with A/B testing

Any new changes or releases you have in the works shouldn’t be driven by your gut feeling but rather from a combination of quantitative and qualitative data, the kind of data and insights that you can only obtain from running tests and experiments. This enables you to optimize your website accordingly rather than speed up losses from a misguided UX change based solely on personal opinion.

A/B testing doesn’t have to be a risky endeavor. Features such as dynamic allocation can make all the difference when running A/B tests to avoid any significant loss in conversions from a poorly performing variation.

In the end, it’s a win-win situation: you get valuable insights while managing risk and your customers receive higher-quality products they actually want resulting in enhanced customer satisfaction.

With AB Tasty, you never have to lose a single conversion. Get started building your A/B tests today with our best-in-class software solution to explore how you can get maximum results with minimal effort thanks to our dynamic allocation capability.

You might also like...

See all

Article

4min read

Beauty E-Commerce Gets a Glow-Up: Insights from Our Cosmetics Consumer Report

Kate Feng

Aug 21, 2025

Article

7min read

Winning Big During the E-Commerce Holiday Season: Strategies and Insights

Brian Heberer

Aug 18, 2025

Article

3min read

Unlock Any Audience Source with AB Tasty’s Universal Connector

Gaetan Postorivo

Aug 1, 2025

Subscribe to
our Newsletter

Article

Dec 5, 2022

12min read

10 Best Practices to Creating Powerful Sign-up Forms

Anthony Brebion

We’ve all been through the pain of filling out never-ending forms where we eventually gave up because it was so complicated.

Simple or complex, sign-up forms are everywhere. You are either the one creating the forms or the one filling them out. From e-commerce to SaaS and media platforms, there is no way to escape them. They’re a part of our everyday digital life.

Because consumers are used to filling out sign-up forms, the smallest mistakes in design or the information you’re asking for can have huge consequences on your conversions.

In this article, we’ll cover the basics, the different types of sign-up forms, and 10 best practices for designing powerful sign-up forms that convert.

What is a sign-up form?

A sign-up form is a webpage, questionnaire, document or popup that visitors must fill out with their personal data in order to access specific content or subscribe to a service.

Sign-up forms can have multiple purposes, but typically share a common goal: acquire personal consumer information, such as their name and email, in exchange for access to top-notch information or services.

For many websites, sign-up forms can also represent the entry point that makes all further conversions possible. This is particularly true for freemium or subscription-based models.

E-commerce websites also rely heavily on these forms because they’re a mandatory step prior to any purchase.

Why do sign-up forms matter so much?

Sign-up forms are where conversions happen. In other words, sign-up forms create business.

In our digital era, forms are how companies can get up close and personal with their potential customers. This is the beginning of their interactions where they collect emails, basic user information, leads and deliver content.

In short, forms are at the center of many digital interactions:

For the service industry, forms are where leads are made.
For the e-commerce industry, forms are where purchases are made.
For the SaaS industry, forms are where you acquire customers.

As you can see, sign-up forms are the central piece of the puzzle.

Knowing this, it’s no surprise that neglecting forms is detrimental to any business. In fact, even a single mistake can promptly cost big companies millions in lost revenues.

For example, travel company Expedia found out that adding one extra line in their registration form had cost them over 12M$ per year, as measured with analytics after correction.

Finding out what’s wrong with a form takes time. It requires your team to test out your sign-up forms using A/B testing to see what is the most appealing for your users.

AB Tasty is an example of an A/B testing tool that allows you to quickly test elements of your sign-up form or different portions of your web page. With AB Tasty’s low-code solution, you can get these tests launched with ease, gather insights via an ROI dashboard, and start increasing your conversions.

4 main types of sign-up forms

1. Email sign-up forms

Emails are a precious touchpoint that shouldn’t be neglected.

These are forms aimed at harvesting email addresses to enhance your email list and generate potential leads.

In the image below, we can see an example of Hubspot using FOMO (fear of missing out) to promote their Service Blog by asking for only one email address.

By keeping the sign-up simple and offering some value through your email content, you’re encouraging your prospects to engage in a short, informational exchange.

2. Product sign-up forms

Product sign-up forms are crucial to e-commerce websites because they’re the last barrier before any purchase is made.

For product sign-up forms, it’s best practice to show the actual product, be very clear, and display security elements to give your customer peace of mind.

While there’s no consensus around the question, we think that e-commerce product forms should be reduced to the bare minimum to decrease the shopping cart abandonment rate.

In any case, delivery and payment options can be separated in order to streamline the checkout process.

3. Subscription sign-up forms

Subscription sign-up forms are a central piece of any subscription-based digital business; it’s where the conversions happen.

However, converting someone into a paying user isn’t always so simple.

Typically, SaaS and subscription-based businesses need time to educate their potential customers, which is why their subscription forms are key for them.

For subscription forms, it’s always important to remember the key information that you’re looking for and to provide value by offering a demo or a free trial.

See how Lancôme increased revenue by 15% by optimizing their account sign-up pathway.

4. Service sign-up forms

Service sign-up forms differ from subscription forms as they do not necessarily bind the user through a subscription.

Service sign-up forms, like Spotify in the image below, are typically aimed at converting a maximum number of visitors into users. For that to happen, one of the best tools that you can use is a social media sign-up process.

Offering users a quick alternative way to register with social media or Google could multiply your conversion rate.

10 Expert Sign-up Forms Best Practices

1. Make it contextual and obvious

Your forms only serve one purpose: to be filled out by your visitors.

For that reason alone, make sure that your forms are easily found on your website with distinctive colors.

However, it’s great to keep in mind that making your sign-up forms obvious doesn’t mean that you should display them everywhere. Context really matters when it comes to asking your visitors for their personal data.

Let’s take a look at Lever in the example below. Lever offers gated content on specific HR subjects that requires you to register. It uses a clear call-to-action and offers a brief summary that helps them “sell” their content before visitors fill out their personal information.

2. Multi-Step vs Single-Step forms

According to VentureHarbour, multi-step forms tend to outperform single-step forms when it comes to lead generation.

Multi-step forms leave a less intimidating impression on visitors because they only ask for sensitive information at the very end of the form. This method generally leads to better results.

However, when it comes to product and subscription sign-up forms, we’d recommend a clear single-step sign-up form as long forms can discourage even the bravest visitors.

3. Keep your forms simple and easy

There’s a lot of debate when it comes to measuring our online attention span but one thing’s for sure: if your online content isn’t attractive and sharp, you’ll lose potential customers.

In fact, your form length mostly depends on the context. Some industries benefit from longer forms because it gives their websites more credibility, while others see a better return from shorter forms.

Here is one example of a truly simple sign-up form:

Asana does an awesome job at keeping the registration process easy – they only ask for your email address in exchange for a free subscription. Of course, Asana’s team knows that more information is required to properly use their solution. However, they wait until the onboarding process to ask for more user information later on.

On a larger scale, several studies report that shorter forms (fewer fields) tend to yield better results (more conversions).

With every additional field, there’s more of a potential to lose leads or customers. For e-commerce stores, our advice is to reduce the number of fields to the bare minimum. Your goal is to offer a slick and swift buying process.

It’s important to keep in mind that a simple form doesn’t mean removing all fields. A simple form means that you only focus on mandatory information that helps you meet your business objectives like the users’ name and email address.

4. Provide some real value

Visitors are asked about their personal information almost everywhere, and data privacy is becoming a prominent concern for many internet users.

With this issue in mind, it’s important to design your sign-up forms in a way that provides value for your visitors in exchange for them filling out the form.

To accomplish this goal, the most popular option is to craft a powerful value proposition to sell your form. This value proposition doesn’t have to be long and detailed – it can be effective by getting straight to the point.

In the example below, outdoor gear specialists REI keeps it very simple with a clear value proposition in their email subform: “co-op offers, events & cool new gear.” When you put your email in the box, you have a very clear idea of what you’re signing up for.

5. Leverage your social proofs

Social proofs help you sell your services and products because it plays on our deeply rooted social nature.

Showing your visitors that many people did the same before gives them comfort in trusting your product or service. This also enhances your brand credibility and helps you achieve higher conversion rates.

It’s all about convincing your visitors to go through the next steps.

In the example below, you can see how Slack leverages social proof by displaying customer testimonials from famous tech companies just above their sign-up form.

6. Make it mobile friendly

As more than half of web traffic comes from mobile devices, mobile friendliness is becoming more important each year.

Knowing that more than half your visitors browse your website with their smartphone, you need to ensure that you pay close attention to your sign-up forms’ mobile layout.

The screen size adaptability of your form could play a decisive role in improving your conversion rate.

7. Don’t ask for password confirmation

Password confirmation typically doesn’t help with conversions. In fact, it slows down the process and actually increases the chances of a user misspelling their password.

Rather than asking for confirmation, allow your visitors to see what they just typed with an icon that unmasks their password. This will give them peace of mind knowing that their password is correct without the frustration of misspelling it.

Avoid password confirmation in your sign-up forms

8. Avoid using Captchas

Although spam is a real issue, you might want to consider avoiding the addition of an anti-spam solution on your visitors’ shoulders.

Captchas, like the one displayed in the image below, can sometimes get messy and irritating, which is why they shouldn’t be overused when it comes to building efficient sign-up forms.

Captchas could have you lose potential customers

However, if your team feels more comfortable having an anti-spam solution, there are better alternatives to captchas to prevent spam.

9. Allow for social sign-ups

As we’ve mentioned a few times in this article, social sign-ups significantly reduce the time it takes for visitors to sign-up thanks to Google or Facebook’s auto-fill.

Implementing social sign-ups makes it easy to subscribe and gives your brand some much-needed credibility. People generally assume that Facebook and Google sign-ups are more secure.

With just one click, it’s an excellent tool to quickly generate leads and improve sign-up conversions.

10. Optimize and test your sign-up forms

Implementing best practices is a good practice in itself. But measuring the changes and their improvements is even better.

Do three fields perform better than six? Is implementing a social media sign-up worth it? Does social proof really give visitors the extra push to fill out your form? Tracking your sign-up form experimentation is the only way to find out.

Your marketing team should test several variations. Using an A/B testing solution, you’ll be able to:

Create assumptions based on your analytics
Test several variations and find a winner
Measure your conversion gains at every stage

Want to get started on A/B testing for your website? AB Tasty is a great example of an A/B testing tool that allows you to quickly set up A/B tests with low code implementation of front-end or UX changes to sign-up forms on your web pages, gather insights via an ROI dashboard, and determine which route will increase your revenue.

Conclusion

As the conversion opportunities for the subscription service industry continue to skyrocket, so does the competition. Online customers are overwhelmed with choices – and often many good ones.

To convert online visitors to your website, you have to have a very consumer-friendly and optimized page. To find what turns your visitors into subscribers, get started on A/B testing your sign-up forms today.

You might also like...

See all

Article

4min read

Beauty E-Commerce Gets a Glow-Up: Insights from Our Cosmetics Consumer Report

Kate Feng

Aug 21, 2025

Article

7min read

Winning Big During the E-Commerce Holiday Season: Strategies and Insights

Brian Heberer

Aug 18, 2025

Article

3min read

Unlock Any Audience Source with AB Tasty’s Universal Connector

Gaetan Postorivo

Aug 1, 2025

Subscribe to
our Newsletter

Article

Nov 28, 2022

7min read

Sample Ratio Mismatch: What Is It and How Does It Happen?

Emily Healy

A/B testing can bring out a few types of experimental flaws.

Yes, you read that right – A/B testing is important for your business, but only if you have trustworthy results. To get reliable results, you must be on the lookout for errors that might occur while testing.

Sample ratio mismatch (SRM) is a term that is thrown around in the A/B testing world. It’s essential to understand its importance during experimentation.

In this article, we will break down the meaning of sample ratio mismatch, how to spot SRM, when it is and is not a problem, why it can happen and how to detect SRM.

Sample ratio mismatch overview

Sample ratio mismatch is an experimental flaw where the expected traffic allocation doesn’t fit with the observed visitor number for each testing variation.

In other words, an SRM is evidence that something went wrong.

Sample ratio mismatch is crucial to be aware of in A/B testing.

Now that you have the basic idea, let’s break this concept down piece by piece.

What is a “sample”?

The “sample” portion of SRM refers to the traffic allocation.

Traffic allocation refers to how the traffic is split toward each test variation. Typically, the traffic will be split equally (50/50) during an A/B test. Half of the traffic will be shown the new variation and the other half will go toward the control version.

This is how an equal traffic allocation will look for a basic A/B test with only one variant:

A/b testing equal traffic allocation

If your test has two variants or even three variants, the traffic will still be allocated equally to each test to ensure that each version receives the same amount of traffic. An equal traffic allocation in an A/B/C test will be split into 33/33/33.

For both A/B and A/B/C tests, traffic can be manipulated in different ways such as 60/40, 30/70, 20/30/50, etc. Although this is possible, it is not a recommended practice to get accurate and trustworthy results from your experiment.

Even by following this best practice guideline, equally allocated traffic will not eliminate the chance of an SRM. This type of mismatch is something that can still occur and must be calculated no matter the circumstances of the test.

Define sample ratio mismatch (SRM)

Now that we have a clear picture of what the “sample” is, we can build a better understanding of what SRM means:

SRM happens when the ratio of the sample does not match the desired 50/50 (or even 33/33/33) traffic allocation
SRM occurs when the observed traffic allocation to each variant does not match the allocation chosen for the test
The control version and variation receive undesired mismatched samples

Whichever words you choose to describe SRM, we can now understand our original definition with more confidence:

“Sample ratio mismatch is an experimental flaw where the expected traffic allocation doesn’t fit with the observed visitor number for each testing variation.”

sample ratio mismatch

Is SRM always a problem?

To put it simply, SRM occurs when one test version receives a noticeably different amount of visitors than what was originally expected.

Imagine that you have set up a classic A/B test: Two variations with 50/50 traffic allocation. You notice at one point that version A receives 10,000 visitors and version B receives 10,500 visitors.

Is this truly a problem? What exactly happened in this scenario?

The problem is that while conducting an A/B test, an extremely strict respect of the allocation scheme is not always 100% possible since it must be random. The small difference in traffic that is noted in the example above is something we would typically refer to as a “non-problem.”

If you are seeing a similar traffic allocation on your A/B test in the final stages, there is no need to panic.

A randomly generated traffic split has no way of knowing exactly how many visitors will stumble upon the A/B test during the given time frame of the test. This is why toward the end of the test duration, there may be a smaller difference in the traffic allocation while the majority (+95%) of traffic is correctly allocated.

When is SRM a problem?

Some tests may have SRM due to experimental setup.

When the SRM is a big problem, there will be a noticeable difference in traffic allocation.

If you see 1,000 directed to one variant and 200 directed to the other — this is an issue. Sometimes, spotting SRM does not require a particular mathematical formula dedicated to calculating SRM as it is evident enough on its own.

However, an extreme difference in traffic allocation can be very rare. Therefore, it’s essential to check the visitor counts in an SRM test before each test analysis.

Does SRM occur frequently?

Sample ratio mismatch can happen more often than we think. According to a study done by Microsoft & Booking, about 6% of experiments experience this problem.

Furthermore, if the test includes a redirect to an entirely new page, SRM can be even more likely.

Since we heavily rely on tests and trust their conclusions to make strategic business decisions, it’s important that you are able to detect SRM as early as possible when it happens during your A/B test.

Can SRM still affect tests using Bayesian?

The reality is that everyone needs to be on the lookout for SRM, no matter what type of statistical test they are running. This includes experiments using the Bayesian method.

There are no exemptions to the possibility of experiencing a statistically significant mismatch between the observed and expected results of a test. No matter the test, if the expected assumptions are not met, the results will be unreliable.

Sample ratio mismatch: why it happens

Sample ratio mismatch can happen due to a variety of different root causes. Here we will discuss three common examples that cause SRM.

One common example is when the redirection to one variant isn’t working properly for poorly connected visitors.

Another classic example is when the direct link to one variant is spread on social media, which brings all users who click on the link directly to one of the variants. This error does not allow the traffic to be properly distributed among the variants.

In a more complex case, it’s also possible that a test including JS code is crashing a variant and therefore some of the visitor configurations. In this situation, some visitors that are being sent to the crashing variant won’t be collected and indexed properly, which leads to SRM.

All of these examples have a selection bias: some non-random visitors are excluded. The non-random visitors are arriving directly from a link shared on social media, have a poor connection, or are visiting a crashing variant.

In any case, when these issues occur, the SRM is an indication that something went wrong and you cannot trust the numbers and the test conclusion.

Checking for SRM in your A/B tests

Something important to be aware of when doing an SRM check is that the priority metric when checking needs to be “users” and not “visitors.” Users are the specific people that are allocated to each variation, meanwhile, the visitors metric is counting the number of sessions that each user makes.

It’s important to differentiate between users and visitors because results may be skewed if a visitor comes back to their variation multiple times. SRM detected with “visitors” may not be the most reliable metric, but using the “users” metric is evidence of a problem.

SRM in A/B testing

Testing for sample ratio mismatch may seem a bit complicated or unnecessary at first glance. In reality, it’s quite the opposite.

Understanding what SRM is, why it happens, and how it can affect your results is crucial in A/B testing. Running an A/B test to help make key decisions is only helpful for your business if you have reliable data from those tests.

Want to get started on A/B testing for your website? AB Tasty is a great example of an A/B testing tool that allows you to quickly set up tests with low code implementation of front-end or UX changes on your web pages, gather insights via an ROI dashboard, and determine which route will increase your revenue.

You might also like...

See all

Article

4min read

Beauty E-Commerce Gets a Glow-Up: Insights from Our Cosmetics Consumer Report

Kate Feng

Aug 21, 2025

Article

7min read

Winning Big During the E-Commerce Holiday Season: Strategies and Insights

Brian Heberer

Aug 18, 2025

Article

3min read

Unlock Any Audience Source with AB Tasty’s Universal Connector

Gaetan Postorivo

Aug 1, 2025

Subscribe to
our Newsletter

Article

Nov 23, 2022

6min read

How AB Tasty Delivers High-Quality Risk-Free Releases with Feature Flags

Rowan Haddad

With their feature flagging functionality, AB Tasty were able to safely and quickly launch new changes to end users without impacting quality through progressive delivery and continuous feedback loops.

In the world of SaaS, velocity and quality are of utmost importance. This is an industry that is constantly evolving and companies must work diligently to keep pace with consumers’ fast-changing needs and to maintain competitive advantage.

AB Tasty has seen a high growth in users all around the world. Consequently, AB Tasty had to accelerate their development processes, which meant that development and feature teams experienced high growth in order to enable the development of more features and increase speed of delivery to market.

The challenges of CI/CD

However, with such high growth and scaling, the company was faced with many growing pains and bottlenecks, which significantly slow down CI/CD processes. This increased the risk of pile up of issues, which defeats the initial purpose of accelerating time-to-market.

Even with mature CI/CD processes, developer and product teams are not immune to pitfalls that impact speed of delivery and go-to-market.

With these challenges in mind, the team at AB Tasty had four main objectives in mind:

Accelerate time-to-market.
Increase speed of delivery without sacrificing quality.
Allow teams to remain autonomous to avoid delays.
Reduce risk by avoiding big bang releases.

The team essentially needed a stress-free solution to push code into production and an easy-to-use interface that can be used by development teams to release features as soon as they’re ready to eliminate the issue of bottlenecks and by product teams to gain more control of the release process by running safe experiments in production to gather useful feedback.

This is when the team at AB Tasty turned to their flagging feature.

Feature flags were a way for the team to safely test and deploy new changes to any users of their choice while keeping them turned off for everyone else.

The team at AB Tasty was able to do this by, first, defining a flag in the feature management interface whose value is controlled remotely by the tool’s API.

The team can then set targeting rules, that is the specific conditions for the flag to be triggered, based on the user ID. Using feature flags, they can perform highly granular user targeting, allowing them to target users using any user attributes to which they have access.

Then, in AB Tasty’s own codebase, teams can simply condition the activation of the feature that interests them, or its behavior, according to the value of the variable, using a simple conditional branch.

At the time, the company was working on a key project to revamp a major part of the UI namely the navigation system, which includes a new vertical navigation and new responsive grids to offer new personalization campaigns with the goal to make it more understandable to users.

For a project of this scope, there were big changes tied to many dependencies, such as the database, and so AB Tasty needed a way to progressively deploy these new changes to obtain initial feedback and avoid a big bang release.

Progressively deliver features

With such a large project, the goal was to mitigate risk by avoiding deploying major changes to all users at once. With feature flags, teams are able to reduce the number of users who can access the new changes.

In particular, the ON/OFF deployment logic of feature toggles within the feature management tool’s interface works like a switch so teams can progressively roll out features based on pre-set targeting rules while turning them off for everyone else.

Easily set up and manage beta and early adopter lists

After testing internally, the product team was looking for a way to easily manage their early adopter list before releasing to the rest of their users. This will enable them to receive quicker feedback from the most relevant (and more forgiving) users.

With AB Tasty’s flagging functionality, teams can simply add these early adopters’ account ID into the targeting of the flag, where they can then immediately access the new feature exclusively before anyone else.

Release without stress by ensuring that developers are ready to tackle any issues

Since most of the development team was based in France, the new changes were initially rolled out to that region so developers can ensure that everything works and can quickly fix any bugs before deploying to other regions (and timezones).

Should anything go wrong, teams can easily roll back the release with a kill switch by immediately toggling a flag off within the feature flagging platform interface so that the feature is no longer visible.

Enable continuous feedback loops

Teams can now test in production on end-users and to optimize features and iterate faster based on real production data. As a result, teams can launch the end-product to all users with the reassurance that they have identified and fixed any issues.

This also empowers teams to become more innovative, as they now have a safe way to test and receive feedback on their ideas, and are no longer limited in their scope of work.

Accelerate go-to-market

Furthermore, the ON/OFF deployment logic allows teams to release at their own pace. This accelerates the time-to-market as developers no longer need to wait for all changes to be ready to release their own changes resulting in less delays and disgruntled customers.

This speed would not be at the expense of quality as with continuous feedback loops, teams can iterate releases which ensures that only high quality products are released to users.

Teams can send features to production whenever they’re ready, make them visible to some users and officially launch to market once all deliverables are ready and good to go!

Download PDF

You might also like...

See all

Article

4min read

Beauty E-Commerce Gets a Glow-Up: Insights from Our Cosmetics Consumer Report

Kate Feng

Aug 21, 2025

Article

7min read

Winning Big During the E-Commerce Holiday Season: Strategies and Insights

Brian Heberer

Aug 18, 2025

Article

3min read

Unlock Any Audience Source with AB Tasty’s Universal Connector

Gaetan Postorivo

Aug 1, 2025

Subscribe to
our Newsletter

Article

Nov 17, 2022

6min read

How to solve real user problems with a CRO strategy

AB Tasty

Catch up on the previous installment of our Customer-Centric Data Series, How to Become a Data-Centric Company, or read the series introduction.

In the next installment of our series on a data-driven approach to customer-centric marketing, we spoke with our partner Raoul Doraiswamy, Founder & Managing Director of Conversionry to understand the flow of a customer-centric experimentation process, and why it is critical to tap into insights from experimentation processes to make better decisions.

What do you find is the biggest gap in the marketing & growth knowledge among brands right now?

Many brands today have the right set of tools such as technology investments, or the right people with marketing expertise. However, brands often face the issue of not knowing how to meet customer needs/how to give their customers what they want whether on their website, app or through digital advertising on the website, app or digital advertising – in other words, how can these brands increase conversions? Raoul identifies the lack of customer understanding to be at the core of this gap and suggests that brands should adopt a customer-centric, customer-driven process that enables a flow of customer insights, complemented by experimentation.

Which key activities deliver the best insights into customer problems?

Raoul believes that to start a strategy that puts the customers at the core, it is important to have the right data-gathering approach to get insights. It’s the foundation of any experimentation program, but can be applied to all marketing channels.

“Imagine you are an air traffic controller. You have multiple screens constantly feeding you where the planes are, or when they might crash into each other. From all these constant insights, the person in front of the screens will have to make the right decisions,” he shares. “However, there are also inconsequential insights such as baggage holders being full – and it is up to the decision-makers to pick out the critical data and make use of them.”

Raoul provides this analogy to liken it to the role of marketing decision-makers, who normally have a dashboard with metrics like revenue, conversion rate, abandoned cart and more. An insights dashboard helps marketers better understand their customers, combining this real-time data with customer feedback from sources like analytics, heatmaps, session recordings, social media comments and user testing. Solid research can be done through a critical analysis of session recordings and user poll forms, and the main takeaways can be fed to this dashboard. How empowering is that for a marketing decision-maker?

Where are the best sources for experimentation ideas?

Raoul asserts that a combination of quantitative and qualitative analysis is key. Heuristic analysis and competitor analysis are also gold when coming up with experimentation ideas. He continues, “Don’t limit yourself to looking at competitors, look at other industries too. For example, for a $90M trade tools client we had to solve the problem of increasing user sign-ins to their loyalty program. By researching Expedia and Qantas, we got the idea to show users points instead of cash to pay for items.” Raoul shares, “Do heat map analysis, look at session recordings, user polls, run surveys to email databases, and user testing. User testing is critical in understanding the full picture.”

After distilling customer problems and coming up with some rough experimentation ideas, the next step is to flesh out your experiment ideas fully. “Going back to the analogy of the Air Traffic Controller, one person on the team is seeing a potential crash but might have limited experience in dealing with this situation. That’s when more perspectives can be brought in by, let’s say, a supervisor, to make a more well-rounded decision. In the same way, when you are ideating, you do not want to just limit it to yourself but rather have a workshop where you discuss ideas with your internal team. If you are working with an agency, you can still have a workshop with both the agency and the client present, or have your CRO team and product team come together to share ideas. This way, you can get multiple stakeholders involved, each of them being able to provide expertise based on their experience with customers,” says Raoul.

Is there value in running data-gathering experiments (as opposed to improving conversion / driving a specific metric)?

“Yes, absolutely,” replies Raoul. “Aligning growth levers with clients every quarter while working with CRO and Experimentation teams on the experimentation process is important. When working towards the goal of increasing conversions, there are KPIs and predictive models to project the goals.

“On the other hand, if the focus of the program is on product feature validation or reducing the risk of revenue due to untested features, there will be a separate metric for that,” he continues. “It is key to have specific micro KPIs for the tests that are running to generate a constant flow of insights, which then allows us to make better decisions.”

In running data-gathering experiments, features such as personalization can be applied which can have a positive impact on the conversions on the website.

What do brands need to get started?

“To begin, you need to start running experiments. Every day without a test is a day lost in revenue!” heeds Raoul. “For marketing leaders who have yet to start running experiments, you can start by pinpointing customer problems, and the flow of insights. To get the insights, you can gather them from Google Analytics, more specifically, by looking at your funnel. Through these insights, identify the drop-off point and observe the Next Page Path, to see where users go next.

“Take for example an eCommerce platform. If the users are dropping off at the product page instead of adding to the cart and moving on to the shipping page, this shows that they are confused about the shipping requirements. This alone can tell you what goes through the user’s mind. Look at heat maps and session recordings to understand the customer’s problems. The next step then is to solve the issue and to do that, you will need an A/B testing platform. Use the A/B testing platform to build tests and launch them as quickly as possible.”

As for established marketing teams who are already doing some testing, Raoul recommends gathering insights and customer problems as they come in every month. “Then to make sense of the data you’ve collected, you need conversion optimization analysts like our experts at Conversionry who are experienced in distilling data down to problems.”

Identifying customer problems is key. If some of the issues your customers encounter stay unaddressed, it could lead to the initiatives flatlining despite months of experimentation. Instead by keeping customer feedback top of mind, you can start designing, development, testing, speak to experience optimization platforms like AB Tasty to build the experiments, then gather insights, and repeat the cycle to see what wins and what doesn’t.

Get started building your A/B tests today with the best-in-class software solution, AB Tasty. With embedded AI and automation, this experimentation and personalization platform creates richer digital experiences for your customers, fast.

You might also like...

See all

Article

4min read

Beauty E-Commerce Gets a Glow-Up: Insights from Our Cosmetics Consumer Report

Kate Feng

Aug 21, 2025

Article

7min read

Winning Big During the E-Commerce Holiday Season: Strategies and Insights

Brian Heberer

Aug 18, 2025

Article

3min read

Unlock Any Audience Source with AB Tasty’s Universal Connector

Gaetan Postorivo

Aug 1, 2025

Subscribe to
our Newsletter

Article

Nov 15, 2022

9min read

52 Email Subject Lines That Work From Fashion and Beauty Brands

AB Tasty

You’ve all heard the saying before: You should never judge a book by its cover. Sounds ridiculous, right? Well, whether you like it or not, that’s exactly how people tend to determine an email’s worth by its subject line.

To give you some perspective, 35% of email recipients open emails based on subject lines alone according to HubSpot, and 21% of subscribers mark boring ones as spam, even if they know they aren’t.

In other words, customers are tough to impress.

If you really want to get noticed and prove you are deserving of their time and attention, supercharge your email subject lines with emotion. Because in the end, no matter how much we rationalize, our buying decisions spring from our emotions.

Think of what a customer fears, dreams of, desires and wants in a product or solution. Use that as a premise of your subject lines to trigger an emotional response.

Some fashion and beauty brands have already been doing consistently well in this area. It’s time you learn the ropes from them.

Fair warning: It’s not a good idea to rely on emotions alone. The purpose of retail brands, like yours, is to sell. So hit a variety of customer touchpoints, including sending personalized emails, because they are 26% more likely to be opened than those without. Even better, they deliver 6X higher transaction rates.

Anyway, let’s dive in…

Email subject lines that induce FOMO

What does FOMO mean? Fear of missing out.

No one likes missing out on time-sensitive deals, including you. Just think of the last time you jumped on an offer because you didn’t want to regret losing out on a good bargain.

In another case, imagine a time when you felt some stress for not having bought something when there was still time, stock, and a good deal.

The same emotional reaction happens to your customers. Whether it’s creating a top-notch holiday campaign, announcing a limited time or a limited stock sale, pick the right set of words like ‘last chance, ‘don’t miss…’, to create urgency. Seeing such offers immediately activates this fear of missing out and drives them to act right away.

It’s exactly why this email from The North Face displays. The simple nudge is enough to get their recipient to grab this one– the last chance to save some money.

The North Face - using FOMO in email subject line

Examples of FOMO-provoking email subject lines:

Limited time offer:

Alice & Olivia: LAST.DAY.DROP.EVERYTHING.
Dropps: Get 15% off, but move quick.
UNIQLO: TICK TOCK! Free shipping ends tonight
Mango: LAST FEW HOURS to enjoy 30% off the entire collection!

Show it’s in demand:

Nykaa: These will sell out in 3…2..⏳
Sorel: These stunners are selling out fast
Olay: Get it before it’s gone!🎁
Marc Jacobs: New to sale and going fast

Imply negative outcomes:

Nykaa: You snooze, you lose.
Revolve: Say goodbye to 65% off 👋
UNIQLO: Get it before it’s gone
Nike: Good stuff in your cart…checking out > missing out

More ideas:

You’ve still got time
Offer expires tonight
The stock’s running out
UH-oh. Your fav styles are (almost) gone
ATTN: This pack will DEFINITELY sell out
Saying goodbye is the toughest
Our top 10 at 10% off for the next 24 hours
LAST chance to save up to $55
<New product> is selling like hot cakes
You’ve ONE day left: FREE shipping +20% off
Tonight only: Your wish list items on sale
LIMITED-time offer on items on your wishlist

Email subject lines that drive curiosity

Can you guess what makes crime drama series binge-worthy? Why we click intriguing Upworthy and BuzzFeed posts? Or why we can’t stop reading novels? Blame our inherent need to close the loop between what we know and what we want to know. Otherwise known as – our curiosity!

This is also exactly why when people receive emails with partial information in the subject lines, they feel the urgent need to click ‘open.’

A perfect example is this email from Nykaa. This cliffhanger technique adds a touch of drama and mystery that is enough to tempt people to see what’s on the other side.

Nykaa - using curiosity in email subject line

Examples of using curiosity in subject lines:

Huckberry: “Want” — everyone
Michael Kors: Give us an inch…
J.Crew: The shorts circuit
TwoThirds: Meet our unique pieces!
Revolve Tomorrow’s outfit forecast
Kate Spade: You’re getting sleepy…
Steve Madden: TORNADO warning
Anthropologie: Let us per-SUEDE you.
Birchbox: We’ve got some ideas for you.
Sephora: Because you need these.
Estée Lauder Online: Why’s everyone obsessed with retinol?
Jimmy Choo: Dreams are made up of these + complimentary global shipping

More ideas:

Not even in your wildest dreams would…
Got a sec? Open this email right away, or…
FIRST time ever on sale
New launches inside
Is THIS what you’ve been waiting for?
You DESERVE this
Don’t buy from us…
Before you regret not buying your favs
Saying goodbye is the toughest
Did we tell you that…
This is personal…

How to personalize and test subject lines to maximize your open rate

By crafting a personalized email with your customer’s name or a point of interest in the subject line, you’re ensuring a connection with your customer.

According to Forbes, 72% of customers ONLY interact with personalized messaging.

Personalization is an important way to build trust with your customer and maintain your relationship with them before, during, and after their purchases. If you’re not using personalization in your email campaigns, you’re likely missing out on a huge opportunity.

Whether you’re looking to personalize your email content to capture customer attention or A/B test your subject lines to determine the best-performing phrase, choosing the right software will help you transform your ideas into reality.

AB Tasty is the complete experience optimization platform to help you create a richer digital experience for your customers — fast. From recommendations to smart search, this solution can help you achieve personalization with ease, experiment with a low-code implementation and revolutionize your brand and product experiences.

Email subject lines that spark happiness

Since you already occupy a sacred space in your customers’ inboxes, why not become a reason for their happiness?

A lot of things make people happy. So right from using humor, wordplay, solving people’s problems, appealing to their vanity, surprising them with freebies and special offers to reminding them there’s good in them. For example, if you can relate their shopping efforts to contributing to a social cause, that’s one idea of reminding them there’s good inside. There are plenty of ideas that you can use to really catch their eyes and make them feel unique.

This email from Fortress of Inca, for example, plays the vanity card. Honest confession: I was at the receiving end and I had to open the email because my fabulous choice in shoes was being appreciated!

Fortress of Inca_ triggering happiness through email subject line

Examples of subject lines focusing on happiness:

Pura Vida: Save the Amazon Rainforest!
Michael Kors: FOOD IS LOVE: Help Us Watch Hunger Stop
Sephora: Your beauty issues solved
Nicole Miller: Button me up, Button me down.
Alice & Olivia: Hot date? We’ve got you.
J.Crew: We don’t do this (MAJOR) sale too often…
JustFab: Your FREE money is waiting. We put $30 in your account.
Pura Vida: Happy birthday to us (gifts for you!)
DSW: Find out how you can make a difference.
Patagonia: School lunch made easy
The North Face: 5 jackets that will have everyone saying…where’d you get them from!

More ideas:

Our birthday treat = gifts for you!
Giving > Getting
It’s true…these will look dapper on you
What to wear for the New Year’s bash
Style secrets you can master in under 10 minutes
Your winter break packing list is HERE!
You’re one click away from shopping complete spring look
Impressed by items in your card: Buy ‘em before they’re gone
You earned what’s INSIDE this email

Email subject lines that create excitement

So far we’ve already established that humans are emotional creatures. Let’s now use excitement to get your sales meter ticking, because chances are that it results in impulse shopping.

Selling to customers is much easier when they’re excited and in touch with their emotions. In a state of excitement, your level of enthusiasm brings you to think and behave differently.

There are a number of ways to get the excitement level up. You can start by highlighting the exclusiveness of your offer, adding social proof (drop names and numbers), capitalizing action verbs to motivate action completion and even using emojis and exclamation marks.

In the email subject line below, team Pura Vida does two things — attract attention and excite the subscriber with the clever use of an emoji, the word ‘holy smokes,’ and a few ‘!

Pura Vida - triggering excitement in email subject line

Examples of Building Excitement in Subject Lines:

J.Crew: What Adam Scott thinks of our suits
Anthropologie: Just for you: the inside scoop
Pura Vida: Special offer (but only for our BFFs)
Olay: Over 50 million sold ❤️
Sorel: Meet our exclusive Frozen 2 boots✨
DSW: Reviewers have spoken: “BEST BOOTIE EVER!”
Steve Madden: Take our shoe stylists’ word for it
Meundies: Well, this is exciting!
Adidas: You’re in. Welcome to Adidas
Revolve: This collab was made for you
theBalm Cosmetics: Welcome to #THEBALMERS community!
Victoria’s Secret: 60% off bras & apparels, in stores only! GO, GO, GO!

More ideas:

Beyonce-approved autumn wear you can’t say no to
Sweet launch discount (for members only!)
Howdie! Your faves are back in stock
Sshh…you’re the FIRST ONE to be seeing this
It’s between you and us ONLY
For your eyes only, because you wanted it
Baby, it’s baccccck!
It’s your birthday! Get 30% off, a FREE gift and free shipping.

Ready to Craft Email Subject Lines That Deliver?

Full disclaimer: It takes a lot of A/B testing and patience to come up with subject lines that stay within the 40-50 characters limit and stir the right emotions. Thankfully, now you’re aware of which emotions to go after and how to invoke emotional decision-making.

So it’s time to get to work! But before you go, and for maximum impact, keep these equally important things in mind:

Match your brand personality with the tone of voice of your subject lines.
Segment your audience to send targeted/personalized emails.
A/B test your email send times.

You might also like...

See all

Article

4min read

Beauty E-Commerce Gets a Glow-Up: Insights from Our Cosmetics Consumer Report

Kate Feng

Aug 21, 2025

Article

7min read

Winning Big During the E-Commerce Holiday Season: Strategies and Insights

Brian Heberer

Aug 18, 2025

Article

3min read

Unlock Any Audience Source with AB Tasty’s Universal Connector

Gaetan Postorivo

Aug 1, 2025

Subscribe to
our Newsletter

Article

9min read

Chaos Engineering 101: How Chaos Brings Order

Rowan Haddad

As we go deeper into digital transformation and as companies move towards large-scale globally distributed systems, the complexity that comes with them increases. This means that failures in these intricate systems become much harder to predict, as opposed to traditional, monolithic systems.

Yet, these failures could result in high costs for teams to repair them; not to mention the painstaking probability of the potential loss of new and existing customers.

The question then is how can we build confidence in the systems that we put into production? How can teams make sure that they’re releasing stable and resilient software that can handle any unpredictable conditions that they’re put into?

This is when teams turn to what is aptly referred to as “chaos engineering”.

What is chaos engineering?

According to the Principles of Chaos, chaos engineering is “the discipline of experimenting on a system in order to build confidence in the system’s capability to withstand turbulent conditions in production.”

In other words, chaos engineering is the process of testing distributed systems to ensure that it can withstand turbulent conditions and unexpected disturbances. Strictly speaking, this is the “chaos” of production.

Chaos engineering is particularly applicable to large-scale, distributed systems. Since such systems are now hosted on globally distributed infrastructures, there are many complex dependencies and moving parts with several points of failure. This makes it harder to predict when an unexpected error will occur.

Due to the unpredictability of these failures of the components of the system, it becomes harder to test for them in a typical software development life cycle.

This is when the concept of chaos engineering came about as a way to predict and test for such failures and uncover hidden flaws within these systems.

In other words, this concept determines the resilience of these systems by identifying their vulnerabilities by carrying out controlled experiments to test for any unpredictable and unstable behavior.

This is done by breaking things on purpose by injecting failure and various types of faults into the system to see how it responds. This will help determine any potential outages and weaknesses in the system.

The ultimate goal of this is a lesson in how to build more resilient systems.

Where does the term come from?

Before we delve any deeper into chaos engineering, it would be helpful to understand where this concept originated.

Chaos engineering started in 2010 when the engineering team at Netflix decided to develop “Chaos Monkey”, which was later made open source, as the team at Netflix migrated from a monolithic architecture to the cloud, deployed on AWS.

For Netflix, this migration to hundreds of microservices brought on a high amount of complexity; therefore, engineers at Netflix were seeking a better approach to prevent sudden outages in the system.

These engineers were mainly looking for a way to disable instances and services within their architecture to ensure that their system can handle such failures with minimal impact on the user experience, allowing them to build a more resilient and reliable architecture.

The idea behind the Chaos Monkey tool was that they would unleash a “wild monkey” to break down individual components in a carefully monitored environment to make sure that a breakdown in this single component wouldn’t affect the entire system.

This, in turn, helped them locate the weaknesses in the system and build automatic recovery plans to address them and alter the system if necessary so that it could easily tolerate unplanned failures in the future.

Afterwards, Chaos Monkey improved and evolved to allow Netflix engineers to more precisely determine failures by testing against more failure states, enhancing the resilience of their system.

From then on, the chaos journey began for Netflix and later on for many organizations dealing with similar distributed systems.

Principles of chaos engineering

We can deduce that chaos engineering involves running experiments to understand how a distributed system behaves when faced with failure.

Unlike other forms of testing, chaos engineering involves experimentation and learning new things about a system by creating a hypothesis and attempting to prove that hypothesis. If it’s not true, this is a chance to learn something new about the system.

Testing, on the other hand, involves making an assumption about a system based on existing knowledge and determining whether it’s true or not by running tests; in other words, the test is conducted based on knowledge of specific properties about the system. The results, therefore, don’t provide new insights or information.

Chaos engineering, for its part, involves exploring scenarios that don’t usually occur during testing designed to gain new knowledge about the system by considering factors that often go beyond the obvious issues that are normally tested for.

The following principles provide a basis on which to run such experiments on your system:

Plan an experiment

The first step involves planning an experiment, where you will need to pinpoint things that could go wrong. This will require gaining an understanding of your system’s normal behavior and determining what constitutes a normal state. Afterwards, you start off by forming a hypothesis of how you think the components of your system will behave in case something goes wrong and then create your control and experimental groups accordingly.

Defining a metric to measure at this stage is useful to gauge the level of normalcy within your system. These could include metrics such as error rates or latency.

Design real-world events

At this stage, you will outline and introduce real-world events that could potentially cause disruptions to your system such as those that occur within hardware or server or any other external event that could lead to outages in your system such as a sudden spike in traffic, hardware failures, network latency or any event that could potentially disrupt the steady state of the system.

Run the experiment

After defining your system’s normal behavior and the events that could disrupt it, experiments can then be run on your system preferably in a production environment to measure the impact of the failure to gain a better understanding of your system’s real-world behavior.

This will also allow you to prove or disprove your hypothesis. The harder it is to cause an outage in the system, the more confident you can be in your system’s resilience.

However, keep in mind that since your experiments are run in production, it’s important to minimize the blast radius in case something goes wrong. This will ensure that any adverse effects are kept at a minimum and if things go smoothly, you can then gradually increase this radius till it reaches full scale. It’s also wise to have a roll back plan if something does go wrong.

Monitor results

The experiment should give you a clear idea of what’s working and of what needs to be improved by looking for a difference between the control and experimental group. Teams can then make the necessary changes as they’re able to identify what led to the outage or disruption to the service, if relevant.

Why we should break things on purpose: Benefits of chaos engineering

We can look at chaos engineering as a sort of safeguard that helps prevent worst case scenarios from happening and impacting the user experience before they actually happen.

Consequently, chaos engineering has a number of benefits.

Increased reliability and resilience

As we’ve already mentioned, running such controlled chaos experiments will help determine your system’s capabilities, thereby preparing the system against unexpected failures.

Information gathered from these experiments can be used to strengthen your system and increase its resilience by locating potential weaknesses and finding ways to resolve them.

In other words, by learning what failure scenarios to prepare for, teams can improve and speed up their response to troubleshooting incidents.

Enhanced user experience

By strengthening your system, it is less likely that it will experience major outages and downtime that could negatively affect the user experience. It allows you to pinpoint issues and problems before they actually become customer pain points.

This will, in turn, result in improved user experience and increased customer satisfaction as you are now releasing high performing, more resilient software.

Reduced revenue loss

By running chaos experiments, companies can prevent lengthy disruptions and outages to the system, which otherwise could lead to losses in revenue as well as high maintenance costs.

Improved confidence in the system

The insights gathered from these experiments can help teams build more resilient and robust systems.

This means that teams, by predicting the unexpected, are prepared for worst-case scenarios, which helps to increase confidence in their systems by having a recovery plan set up for such scenarios.

Nonetheless, organizations should still carefully consider the challenges of chaos engineering before implementing it as, despite its benefits, it can also be costly and time-consuming.

Unleashing chaos for better digital experiences

As we’ve seen, chaos engineering is an essential practice when it comes to creating uninterrupted, seamless digital experiences for your customers.

It’s not just breaking things for the sake of breaking things; it’s a way to gain insight on how a system behaves and to gauge its resilience. In other words, chaos engineering is not only breaking things, but it’s also about fixing weaknesses in a system to build its resilience by exposing hidden threats thereby minimizing risk.

It’s important to note that chaos engineering isn’t meant to replace the other types of testing that are carried out throughout the software development life cycle but instead to complement these tests to provide a high performing system.

Finally, chaos engineering has an important role in DevOps. At the heart of DevOps is the idea of continuous improvement, which is why integrating chaos engineering into a DevOps methodology is essential to mitigate security risks. It’s also a way for DevOps teams to deal with the rising complexity of applications nowadays.

Consequently, introducing chaos experiments into your DevOps CI/CD pipeline will help teams detect hidden issues more quickly, which grows confidence in the system enabling them to deploy faster to end-users.