Anaïs Levy shares how luxury brands use experimentation to balance brand image with business performance, plus why removing prices can actually boost conversions.
Anaïs Levy knows the secret behind what drives luxury digital experiences. As the E-commerce & Omnichannel Services Insights Manager at the Kering Group, the powerhouse group behind iconic luxury brands such as Gucci, Balenciaga, and Saint Laurent to name a few, she leverages over ten years in conversion rate optimization to help some of the largest luxury brands build dazzling digital experiences.
Before joining Kering, Anaïs worked across various industries, from travel giant Expedia to luxury group LVMH. Her unique position involves analyzing business performance across multiple luxury brands, helping them make data-driven decisions while respecting their distinct artistic visions. As a long-term AB Tasty customer and frequent speaker at industry events, Anaïs brings a rare perspective on how luxury brands navigate the delicate balance between brand image and business optimization.
Anaïs Levy spoke with AB Tasty’s Head of Marketing and host of “The 1000 Experiments Club” podcast, John Hughes, about navigating creative constraints in luxury experimentation, leveraging cross-brand benchmarking to drive results, and how omnichannel thinking is revolutionizing conversion rate optimization in the luxury space.
Here are some of the takeaways from their conversation.
Beyond the website: Luxury’s omnichannel reality
The numbers tell a story that completely reframes how we should think about luxury e-commerce: “Among all our visitors, if we sum up visitors in the stores and visitors on our websites, the majority, like 90% is coming to the website and 10% is our traffic in store,” Anaïs shares.
But here’s the twist—most purchases still happen offline. This flips traditional CRO thinking on its head.
“When you have this overview of understanding the business, understanding how luxury websites fit into the whole customer journey, conversion rate optimization is about understanding how you make the most of each asset you have,” she explains.
For luxury brands, websites serve multiple purposes beyond direct sales. Customers use them for research and discovery, to prepare for store visits, for aspirational browsing, and to access omnichannel services like click-and-collect or appointment booking.
“We know we have aspirational customers, but we were not treating them as a specific segment. So everybody coming to the website should in the end convert. And I guess now with the rise of omnichannel and services… we have come to this conclusion that a significant amount of traffic is not going to purchase online,” Anaïs notes.
The takeaway? Stop measuring luxury e-commerce success purely on online conversion rates. Instead, it’s about thinking bigger. How does your digital experience drive overall brand engagement and omnichannel revenue?
Discover the luxury industry’s glittering guide to all things optimization.
Working with luxury brands means constant negotiation between artistic vision and business performance. “It’s a lot of compromise,” Anaïs admits. “The brand image, the design is really the voice of the artistic director.”
But here’s where persistence pays off. When faced with a creative “no,” Anaïs doesn’t give up—she waits, gathers more data, and asks again. “You have to be stubborn because it could be a no. But two months, three months, six months after you ask again and one day you would have a yes,” she explains.
Her secret weapon? Benchmark data across sister brands. When one Kering brand achieves better checkout completion rates than another, it becomes harder to argue against proven improvements.
“If your sister brands can achieve these figures and they have kind of the same backbone, the same services, same offer, it means that there is something that we are not doing right,” she points out.
This creates a unique advantage where luxury brands can iterate on proven concepts while maintaining their distinct identities. Anaïs’s team has built AB test catalogs shared across all brands and runs group-wide experimentation events to facilitate this knowledge sharing.
Think global, analyze local with segmentation
Anaïs’s team runs experiments globally but analyzes results with surgical precision. “The tip I would give to people who would listen to this podcast is really think global, because then if you make it work for most of your users, your gains are way higher,” she advises.
But the magic happens in the analysis. “When you analyze, don’t forget to try looking at some important segments. So countries could be some… We are using a lot EmotionsAI segmentation to analyze the results because it gives you ideas about why the ‘Competition’ segment didn’t like these experiments,” she explains.
This granular approach reveals opportunities for personalization. By breaking down results by country, device, and behavioral and emotional segments, teams uncover insights that would be invisible in aggregate data.
The strategy works because it balances efficiency with insight: global rollouts maximize impact and streamline development, while segmented analysis reveals why certain groups respond differently, creating opportunities for follow-up experiments that target specific segments with tailored experiences.
What else can you learn from our conversation with Anaïs Levy?
The surprising price experiment: How removing prices from product listing pages actually increased conversions by focusing attention on products rather than cost
AI’s luxury future: From productivity tools to conversational search that mimics in-store personal shopping experiences
The newsletter discovery: How a failed lazy-loading test accidentally revealed hidden engagement opportunities in page footers
Cross-brand collaboration: The internal tools and processes that help luxury brands share learnings while maintaining their unique identities
About Anaïs Levy
Anaïs Levy has over ten years of experience in conversion rate optimization, spanning industries from travel (Expedia) to luxury (LVMH, Kering). At Kering Group, she manages business performance and insights across multiple luxury brands, including Gucci, Saint Laurent, and Balenciaga. Her unique role involves balancing data-driven optimization with the creative constraints of luxury brand management, making her a sought-after speaker on experimentation in highly regulated creative industries.
About 1,000 Experiments Club
The 1,000 Experiments Club is an AB Tasty-produced podcast hosted by John Hughes, Head of Marketing at AB Tasty. Join John as he sits down with the experts in the world of experimentation to uncover their insights on what it takes to build and run successful experimentation programs.
If you’ve ever wondered what to test next, struggled to get developer time, or felt overwhelmed by reporting dashboards, you’re not alone.
These are the frustrations experimentation teams face daily. That’s why we built AB Tasty AI—a suite of AI designed not to add hype to your workflow, but to genuinely help you move faster, test smarter, and get real business impact from your experimentation program.
With AB Tasty AI, those roadblocks disappear. Our AI guides you through ideation, building, personalization, and analysis—so you can focus less on the “what ifs” and more on the results that matter.
Let’s walk through how it works.
AI that crushes your “We’re guessing what to test next” problem
Step 1: Ideation generation
Every experimentation program begins with the same hurdle: what should we test next?
In many organizations, idea generation depends on gut feelings or endless whiteboard sessions that rarely produce actionable outcomes. That’s where AB Tasty AI steps in.
Our platform scans your pages and surfaces data-backed test ideas that are proven to make an impact. Instead of guessing, you get a prioritized list of opportunities aligned with your business goals. It’s like having an intelligent co-pilot who not only brainstorms with you but also brings evidence to the table.
AI that eliminates your “Our hypotheses are hunches” frustration
Step 2: Develop a hypothesis
A test idea is only as strong as the hypothesis behind it. Yet many teams struggle to move from fuzzy thinking to clear, structured hypotheses with measurable objectives.
AB Tasty AI eliminates the guesswork by helping you sharpen your hypotheses. You can turn casual “what if we tried this?” conversations into formal statements that define the change, predict the impact, and set up the right metrics for evaluation.
This structured approach not only improves your test quality but also boosts team confidence and stakeholder trust.
AI that annihilates your “I can’t build what I’m thinking” roadblock
Step 3: Start building
One of the biggest blockers in experimentation is the dependency on developer resources. Great ideas often languish in backlogs because the dev team is focused on other priorities.
With AB Tasty AI, you can instantly transform ideas into buildable experiments—no coding required. Whether you want to tweak a button, test a new layout, or launch a more complex variation, our AI makes it possible to build, preview, and launch without waiting weeks for a developer.
This shift not only accelerates testing velocity but also democratizes experimentation, empowering marketers, product managers, and designers to run with their ideas.
AI that ends your “Our personalization feels robotic” paralysis
Step 4: Understand your audience
Many brands struggle with personalization that feels forced, generic, or robotic. Visitors sense it, and the results often disappoint.
AB Tasty AI introduces EmotionsAI Insights, giving you a window into the emotional triggers that shape customer behavior. Instead of relying only on demographic or behavioral data, you get deeper visibility into what truly motivates your audience.
It’s personalization with empathy—designed to feel natural, human, and meaningful.
AI that solves your “I don’t know why visitors convert” mystery
Step 5: Personalize the customer journey
Understanding emotional drivers is just the start. With EmotionsAI Segments, you can act on those insights by creating experiences tailored to specific motivations.
For example, one group of visitors might be motivated by security and reassurance, while another thrives on novelty and excitement. AB Tasty AI combines emotional, behavioral, and contextual data to reveal these distinctions, allowing you to craft experiences that resonate at a deeper level.
The result? More conversions, stronger loyalty, and a customer journey that feels less like a funnel and more like a personalized conversation.
AI that crushes your “I don’t understand this report” problem
Step 6: Analyze your reports
Once experiments are running, the next challenge is often reporting. Traditional dashboards can be dense, and interpreting results takes time—especially if stakeholders want quick answers.
AB Tasty AI simplifies the process with natural language analysis. You can ask plain-English questions like “Which variation performed best with mobile visitors?” and get clear, actionable answers instantly.
This not only saves hours of manual analysis but also democratizes data, empowering non-technical teams to explore results with confidence.
Why AB Tasty AI Stands Out
The market is full of AI solutions, many of which promise more than they deliver. AB Tasty AI is different. We’ve designed it to remove the real blockers experimentation teams face every day:
No more guessing what to test
No more hunch-based hypotheses
No more dev backlog bottlenecks
No more robotic personalization
No more confusing reports
No more lost learnings
In short, AB Tasty AI moves your experiments from start to success.
FAQs about AI in digital experimentation
What type of AI does AB Tasty offer?
AB Tasty offers practical, experimentation-focused AI that supports the full testing journey. This includes AI for idea generation, hypothesis creation, no-code experiment building, emotional personalization (EmotionsAI), natural language reporting, and more.
How does AB Tasty AI help with personalization?
AB Tasty AI uses EmotionsAI to uncover visitor motivations and segment audiences based on emotional, behavioral, and contextual data. This allows businesses to create experiences that feel more human and relevant.
Can AB Tasty AI help non-technical teams run experiments?
Yes. AB Tasty AI empowers marketers, product managers, and designers to launch tests without relying on developers, thanks to its no-code experiment builder.
What makes AB Tasty AI different from other AI solutions on the market?
AB Tasty AI is designed to deliver practical, business-ready solutions. While many AI tools focus on hype, AB Tasty AI helps teams move from “what if” to “what works” by providing tangible results at every stage of the experimentation cycle.
The way consumers shop for cosmetics is evolving fast. Today’s beauty buyers aren’t just looking for the right shade or texture. They care about what’s inside, how it’s made, and whether they can trust the brand behind it.
To help brands stay ahead, we recently hosted a webinar inspired by our e-book, Decoding Online Shopping: Cosmetics Consumer Trends for 2025. Our hosts, Lara Hourquebie and Justin Trout unpacked what today’s beauty shoppers expect, the digital experiences that build loyalty, and practical test ideas you can apply right away. If you missed the live session, here’s your recap of the new rules shaping beauty e-commerce.
What’s shaping beauty e-commerce in 2025
From our research and client insights, three big themes stood out:
🌱 Sustainability isn’t optional: eco-friendly, cruelty-free, and ethical sourcing have become the baseline.
✨ Ingredient transparency: shoppers want to know exactly what goes into their skincare and cosmetics.
📱 Social media’s influence: skincare routines and beauty standards are amplified online, fueling demand for authenticity and trust.
And yes, price still matters, but high-quality reviews are the second most influential factor.
One of the strongest insights from both the e-book and webinar was the importance of social validation. Shoppers feel reassured when they see that others have purchased, rated, or recommended a product – especially in beauty, where confidence is key.
Clarins put this into practice by experimenting with a social proof widget on their product pages. The idea was simple: show shoppers in real-time that others were also browsing or buying the same product.
The impact?
+5% increase in average order value
+€5.8K uplift in revenue
By targeting this experiment to the right audience segments, Clarins proved that even small nudges can build confidence and boost sales.
Your 2025 beauty brand checklist
Embed sustainability and ingredient transparency into your brand story.
Make reviews and social proof highly visible – don’t leave trust-building to chance.
Test new ideas, even small ones like Clarins’ widget – they can create outsized results.
Keep the focus on loyalty over discounts: long-term trust beats short-term price cuts.
Building better experiences through relevance
As our research shows, shoppers are happy to share details like their skin type, concerns, or makeup preferences if it helps them find the perfect match. But when it comes to things like personal contact details, they’re far less willing.
The takeaway? Consumers want relevance, not noise. They’re open to sharing what improves their journey – as long as brands use it thoughtfully and transparently.
In short: the beauty brands that blend values, personalization, and experimentation will be the ones to win hearts (and baskets) in 2025.
The holiday season is more than just a date on the calendar. For e-commerce teams, it’s the biggest moment of the year—a time of high stakes, fierce competition, and incredible opportunity. Black Friday and Cyber Monday can feel like a mad dash to the finish line, but the teams that win big don’t just show up; they prepare.
The numbers tell the story. In 2024, Black Friday online sales hit $10.8 billion, while Cyber Monday remained the biggest day for online shopping with a staggering $13.3 billion in sales according to Adobe. But here’s what those numbers don’t show: the months of preparation, testing, and collaboration that made those wins possible.
This isn’t about finding a single “hack” for holiday success. It’s about building momentum through smart, steady, and collaborative effort. It’s about asking the right questions, testing bold ideas, and learning from every single click.So, let’s get ready together. Here are some proven strategies—many of which you can find in our “30 Tests for Black Friday” e-book—to help your team go further this season.
First things first: Build a foundation of trust
Before the first sale banner goes up, your site needs to be ready for the rush. A slow or buggy experience is the fastest way to lose a customer. The difference between a page that loads in one second and a page that loads in five seconds can be a 17% drop in conversion rate.
Start by pressure-testing your site. Simulate high-traffic moments to find and fix potential bottlenecks before they become holiday headaches. Compress your images, streamline your code, and make sure your servers can handle the peak. A smooth, reliable experience is non-negotiable—it’s the foundation of trust.
Create urgency that doesn’t just shout, it connects
The holidays run on urgency, but “BUY NOW!” only gets you so far. The real goal is to create a sense of shared excitement and opportunity. This is where testing your messaging becomes essential.
Try moving beyond generic phrases. Experiment with different calls-to-action (CTAs), placements, and designs to see what truly motivates your audience.
Promotional Banners: Don’t assume every visitor lands on your homepage. MeUndies added a bold promo-code banner to their product detail pages (PDPs) to catch visitors arriving from ads. This small change led to a 9.8% increase in transactions and an uplift of over $400,000 in revenue.
Countdown Timers: The team at Moulinex wanted to drive traffic to a limited-time shopping event. They tested a simple banner with a countdown timer against one without. The result? The countdown banner drove 74% more clicks on the CTA. It’s a simple way to show that time to grab a great deal is running out.
Make every experience feel personal
Holiday shoppers aren’t a monolith. They’re individuals with unique tastes and needs, and they expect to be treated that way. Personalization is how you show your customers that you see them. Use browsing behavior and purchase history to tailor offers that feel less like a sales pitch and more like a helpful suggestion.
For example, if a customer has been eyeing your winter coats, feature them prominently. Dynamic content that adapts to user behavior makes every visit feel relevant and keeps people engaged.The team at Pan Pacific Hotels Group took this to heart. They created two audience segments—families and couples—and tailored the entire website experience with custom banners and offers. The result was a 35% uplift in bookings from these visitors. That’s the power of making it personal.
Non-negotiable: optimize for mobile experiences
More than half of all e-commerce sales happen on a phone. This isn’t a trend; it’s the standard. A clunky mobile experience isn’t just an annoyance—it’s a dealbreaker.
Your mobile site needs to be fast, responsive, and incredibly easy to navigate. Think simplified checkouts and thumb-friendly buttons. Test every step of the mobile journey, from landing page to confirmation.
A great idea is to add a fixed search bar for mobile visitors. When the Calvin Klein team tested this, they saw a 267% increase in search bar clicks. It’s a small change that removes friction and helps people find what they’re looking for, faster.
Build confidence with social proof
In a sea of holiday deals, trust is your most valuable currency. Social proof—like customer reviews, ratings, and real-time activity—is a powerful way to build that trust. It shows hesitant shoppers that they’re in good company.
Showcase what’s popular:NYX Professional Makeup tested adding a message on their product pages that said, “75 beauties have purchased this product today!” This simple addition doubled their transaction rate.
Suggest scarcity:L’Occitane en Provence tested showing how many people were currently viewing a product. This touch of social proof increased their transaction rate by 5.8%.
When people see that others are buying and loving your products, it makes their decision to purchase that much easier.
A checkout that just works
Cart abandonment is the ghost of holidays past, present, and future. A complicated, clunky checkout process is often the culprit. Your mission is to make paying as simple and seamless as possible.
Minimize the steps, offer multiple payment options (like PayPal or Apple Pay), and be upfront about shipping costs. Every bit of friction you remove is a potential sale saved.
The team at Galeries Lafayette noticed that two-factor authentication was adding an extra step to checkout. They tested a version that prioritized payment methods without it. The change was a huge win, boosting their conversion rate by 38%. It proves that even small steps toward simplicity can lead to big progress.
Keep the conversation going
The holiday season is an amazing time for acquiring new customers, but the real win is turning them into loyal fans who stick around. The journey doesn’t end when the package arrives.
Think about what comes next.
A simple, personalized thank-you email.
An exclusive offer for their next purchase.
An invitation to join your loyalty program.
Building these post-holiday touchpoints is how you turn a seasonal rush into sustainable growth. It’s about building a relationship, not just completing a transaction.
Your holiday prep list: Prepare, test, iterate
Winning the holiday season is a marathon, not a sprint. It’s the result of the work you put in beforehand—the preparation, the testing, and the courage to try something new.
By focusing on a solid foundation, personalizing the experience, and removing friction at every turn, you’re not just optimizing a website; you’re building a better journey for your customers.
Stay agile, keep learning, and trust in the process. With your team behind you and a willingness to iterate, you can do more than just survive the holiday rush—you can thrive in it.
Breaking down silos between your data and your experiments
Every marketing team dreams of having a unified view of their customers. But in reality, data often lives in silos: a CRM here, a CDP there, a custom analytics tool somewhere else. If you want to use these audience segments inside AB Tasty for targeting and personalization, you need a simple way to connect them to our platform—regardless of the tool you use.
That’s exactly what AB Tasty’s Universal Connector delivers.
What is the Universal Connector?
The Universal Connector is built on top of AB Tasty’s Universal Data Connector (UDC). It allows you to import audience data from any third-party tool—even those not available as native integrations—and make them available in AB Tasty’s Segment Builder for experiments, personalizations, and patches.
In simple terms: If your tool can send audience data to AB Tasty, the Universal Connector can make it actionable.
Why it matters
Agnostic by design: Works with any CRM, CDP, analytics platform or custom tool capable of sending data via API.
Self-service: Marketers can set up their connectors through a simple interface—no heavy dev work required.
Unified audience view: Imported audiences automatically appear in AB Tasty’s audience management and, once synced, are instantly available in the Segment Builder—ready to power your campaigns.
Compatible with BYID: Works seamlessly with AB Tasty’s Bring Your Own ID feature, ensuring perfect reconciliation with your own user IDs across devices and platforms.
From complex workflows to a streamlined process
Traditional approach:
Importing custom audiences often requires manual code injection, custom attributes, and support from technical teams.
Segments have to be declared one by one in the JavaScript console, with a risk of errors.
With AB Tasty’s Universal Connector:
Configure a connector in a few clicks: declare the source, specify how to identify the visitor (cookie, local storage, BYID), and let AB Tasty handle the reconciliation automatically.
Audiences flow directly into the Segment Builder without manual coding.
Who benefits the most?
Global brands with complex stacks: Multiple CRMs, custom CDPs, or proprietary data systems.
The Travel & Hospitality industry: Hotel groups, cruise lines, and booking platforms that need to unify CRM, loyalty, and reservation data across web and mobile apps.
The Retail & e-commerce sector: Brands that want to leverage loyalty IDs or offline CRM segments in their onsite personalizations.
A powerful use case: Travel & Hospitality
Travel brands often need to merge data from several tools: CRM, booking engines, loyalty programs, and mobile apps. The Universal Connector makes it easy to bring these audiences into AB Tasty and deliver hyper-personalized experiences.
For example, a major international hotel group uses the connector to unify audiences from its CDP and CRM, enabling precise targeting based on booking history and loyalty status across devices.
Getting started
The Universal Connector is designed to be quick to set up and easy to maintain. Most of the work can be done by a marketer, with only light support from a technical contact.
Create your connector: Make sure the audience identifier matches your imported file and specify how visitors are identified—via cookie, localStorage, or your own ID (BYID). The connector will handle the rest.
Send your audience data to UDC: Push your segments via a simple API call.
Target with confidence: Once synced, your imported segments automatically appear in the Segment Builder, ready to use in experiments and personalizations.
Bonus point: No complex coding. No manual segment declarations. Just a straightforward connection between your data and AB Tasty.
Ready to unlock your audiences?
Go beyond native integrations and make any audience actionable with AB Tasty’s Universal Connector.
Contact your KAM and start importing any audience into AB Tasty today.
If you’ve ever run an A/B test, you know the thrill of watching those numbers tick up and down, hoping your new idea will be the next big winner. But behind every successful experiment is a secret ingredient: the statistical model that turns your data into decisions.
With so many options – Bayesian, Frequentist, CUPED, Sequential – it’s easy to feel like you’re picking a flavor at an ice cream shop you’ve never visited before. Which one is right for you? Let’s dig in!
The Scoop on Statistical Models
Statistical models are the brains behind your A/B tests. They help you figure out if your shiny new button color is actually better, or if you’re just seeing random noise. But not all models are created equal, and each has its own personality – some are straightforward, some are a little quirky, and some are best left to the pros.
Bayesian Testing Model: The Friendly Guide
Imagine you’re asking a friend, “Do you think this new homepage is better?” The Bayesian model is that friend who gives you a straight answer: “There’s a 92% chance it is!” Bayesian statistics use probability to tell you, in plain language, how likely it is that your new idea is actually an improvement.
Bayesian analysis works by updating what you believe as new data comes in. It’s like keeping a running tally of who’s winning the race, and it’s not shy about giving you the odds. This approach is especially handy for marketers, product managers, and anyone who wants to make decisions without a PhD in statistics. It’s clear, actionable, and – dare we say – fun to use.
At AB Tasty, we love Bayesian. It’s our go-to because it helps teams make confident decisions without getting tangled up in statistical spaghetti. Most of our clients use it by default, and for good reason: it’s easy to understand, hard to misuse, and perfect for fast-paced digital teams.
Pros of Bayesian Testing:
Results are easy to interpret (“There’s a 92.55% chance to win!”).
Great for business decisions (and no need to decode cryptic p-values).
Reduces the risk of making mistakes from peeking at your data.
Cons of Bayesian Testing:
Some data scientists may prefer more traditional methods.
Can require a bit more computing power for complex tests.
Frequentist Testing Model: The Classic Statistician
If Bayesian is your friendly guide, Frequentist is the wise professor. This is the classic approach you probably learned about in school. Frequentist models use p-values to answer questions like, “If there’s really no difference, what are the chances I’d see results like this?”
Frequentist analysis is all about statistical significance. If your p-value is below 0.05, you’ve got a winner. This method is tried and true, and it’s the backbone of academic research and many data teams.
But here’s the catch: p-values can be tricky. They don’t tell you the probability that your new idea is better; they tell you the probability of seeing your data if nothing is actually different. It’s a subtle distinction, but it trips up even seasoned pros. If you’re comfortable with statistical lingo and want to stick with tradition, the Frequentist model is a good choice. Otherwise, it can feel a bit like reading tea leaves.
Pros of Frequentist Testing:
Familiar to statisticians and data scientists.
Matches legacy processes in many organizations.
Cons of Frequentist Testing:
Results can be confusing for non-experts.
Easy to misinterpret, leading to “false positives” if you peek at results too often.
CUPED Testing Model: The Speedster (But Only for the Right Crowd)
CUPED (Controlled Experiment Using Pre-Experiment Data) is designed to go fast by using data from before your experiment even started. By comparing your test results to users’ past behavior, CUPED can reduce the noise and help you reach conclusions quicker.
But here’s the twist: CUPED only shines when your users come back again and again, like on streaming platforms (Netflix) or big SaaS products (Microsoft). If you have an e-commerce site, CUPED can actually steer you wrong, leading to misleading results.
For most e-commerce teams, CUPED is a bit like putting racing tires on a city bike, not the best fit. But if you’re running experiments on a platform with high user recurrence, it can be a powerful tool in your kit.
Pros CUPED Testing:
Can deliver faster, more precise results for high-recurrence platforms.
Makes the most of your existing data.
Cons of CUPED Testing:
Not suitable for most e-commerce or low-frequency sites.
Can lead to errors if used in the wrong context.
More complex to set up and explain.
Sequential Testing Model: The Early Warning System
Sequential testing is your experiment’s smoke alarm. Instead of waiting for a set number of visitors, it keeps an eye on your results as they come in. If things are going south – say, your new checkout flow is tanking conversions – it can sound the alarm early, letting you stop the test and save precious traffic.
But don’t get too trigger-happy. Sequential testing is fantastic for spotting losers early, but it’s not meant for declaring winners ahead of schedule. If you use it to crown champions too soon, you risk falling for false positives – those pesky results that look great at first but don’t hold up over time.
At AB Tasty, we use sequential testing as an early warning system. It helps our clients avoid wasting time and money on underperforming ideas, but we always recommend waiting for the full story before popping the champagne.
Pros of Sequential Testing:
Helps you spot and stop losing tests quickly.
Saves resources by not running doomed experiments longer than necessary.
Cons of Sequential Testing:
Not designed for picking winners early.
Can lead to mistakes if used without proper guidance.
Which Statistic Model is Best for A/B Testing?
If you’re looking for a model that’s easy to use, hard to misuse, and perfect for making fast, confident decisions, Bayesian is your best bet – especially if you’re in e-commerce or digital marketing. It’s the model we recommend for most teams, and it’s the default for a reason.
If you have a team of data scientists who love their p-values, or you’re working in a highly regulated environment, Frequentist might be the way to go. Just be sure everyone’s on the same page about what those numbers really mean.
Running a streaming service or a platform where users log in daily? CUPED could help you speed things up – just make sure you’ve got the right data and expertise.
And if you want to keep your experiments safe from disasters, Sequential is the perfect early warning system.
Conclusion: The Right A/B Testing Model for the Right Job
Choosing a statistical model for A/B testing doesn’t have to be a headache. Think about your team, your users, and your goals. For most, Bayesian is the friendly, reliable choice that keeps things simple and actionable. But whichever model you choose, remember: the best results come from understanding your tools and using them wisely.
Ready to run smarter, safer, and more successful experiments? Pick the model that fits your needs—and don’t be afraid to ask for help if you need it. After all, even the best chefs need a good recipe now and then.
Average Order Value (AOV) is a widely used metric in Conversion Rate Optimization (CRO), but it can be surprisingly deceptive. While the formula itself is simple—summing all order values and dividing by the number of orders—the real challenge lies within the data itself.
The problem with averaging
AOV is not a “democratic” measure. A single high-spending customer can easily spend 10 or even 100 times more than your average customer. These few extreme buyers can heavily skew the average, giving a limited number of visitors disproportionate impact compared to hundreds or thousands of others. This is problematic because you can’t truly trust the significance of an observed AOV effect if it’s tied to just a tiny fraction of your audience.
Let’s look at a real dataset to see just how strong this effect can be. Consider the order value distribution:
The horizontal axis represents the order value.
The vertical axis represents the frequency of that order value.
The blue surface is a histogram, while the orange outline is a log-normal distribution approximation.
This graph shows that the most frequent order values are small, around €20. As the order value increases, the frequency of such orders decreases. This is a “long/heavy tail distribution,” meaning very large values can occur, albeit rarely.
A single strong buyer with an €800 order value is worth 40 times more than a frequent buyer when looking at AOV. This is an issue because a slight change in the behavior of 40 visitors is a stronger indicator than a large change from one unique visitor. While not fully visible on this scale, even more extreme buyers exist.
The next graph, using the same dataset, illustrates this better:
The horizontal axis represents the size of the growing dataset of order values (roughly indicating time).
The vertical axis represents the maximum order value in the growing dataset in €
At the beginning of data collection, the maximum order value is quite small (close to the most frequent value of ~€20). However, we see that it grows larger as time passes and the dataset expands. With a dataset of 10,000 orders, the maximum order value can exceed €5,000. This means any buyer with an order above €5,000 (they might have multiple) holds 250 times the power of a frequent buyer at €20. At the maximum dataset size, a single customer with an order over €20,000 can influence the AOV more than 2,000 other customers combined.
When looking at your e-commerce metrics, AOV should not be used as a standalone decision-making data.
E-commerce moves fast. Get the insights that help you move faster. Download the 2025 report now.
The problem intensifies when considering the random splits used in A/B tests.
Imagine you have only 10 very large spenders whose collective impact equals that of 10,000 medium buyers. There’s a high probability that the random split for such a small group of users will be uneven. While the overall dataset split is statistically even, the disproportionate impact of these high spenders on AOV requires specific consideration for this small segment. Since you can’t predict which visitor will become a customer or how much they will spend, you cannot guarantee an even split of these high-value users.
This phenomenon can artificially inflate or deflate AOV in either direction, even without a true underlying effect, simply depending on which variation these few high spenders land on.
What’s the solution?
AOV is an unreliable metric, how can we effectively work with it? The answer is similar to how you approach conversion rates and experimentation.
You don’t trust raw conversion data—one more conversion on variation B doesn’t automatically make it a winner, nor do 10 or 100. Instead, you rely on a statistical test to determine when a difference is significant. The same principle applies to AOV. Tools like AB Tasty offer the Mann-Whitney test, a statistical method robust against extreme values and well-suited for long-tail distributions.
AOV behavior can be confusing because you’re likely accustomed to the more intuitive statistics of conversion rates. Conversion data and their corresponding statistics usually align; a statistically significant increase in conversion rate typically means a visibly large difference in the number of conversions, consistent with the statistical test. However, this isn’t always the case with AOV. It’s not uncommon to see the AOV trend and the statistical results pointing in different directions. Your trust should always be placed in the statistical test.
The root cause: Heavy tail distributions
You now understand that the core issue stems from the unique shape of order value distributions: long-tail distributions that produce rare, extreme values.
It’s important to note that the problem isn’t just the existence of extreme values. If these extreme values were frequent, the AOV would naturally be higher, and their impact would be less dramatic because the difference between the AOV and these values would be smaller. Similarly, for the splitting problem, a larger number of extreme values would ensure a more even split.
At this point, you might think your business has a different order distribution shape and isn’t affected. However, this shape emerges whenever these two conditions are met:
You have a price list with more than several dozen different values.
Visitors can purchase multiple products at once.
Needless to say, these conditions are ubiquitous and apply to nearly every e-commerce business. The e-commerce revolution itself was fueled by the ability to offer vast catalogues.
Furthermore, the presence of shipping costs naturally encourages users to group their purchases to minimize those costs. It means that nearly all e-commerce businesses are affected. The only exceptions are subscription-based businesses with limited pricing options, where most purchases are for a single service.
Here’s a glimpse into the order value distribution across various industries, demonstrating the pervasive nature of the “long tail distribution”:
Cosmetic
Transportation
B2B packaging (selling packaging for e-commerce)
Fashion
online flash sales
AOV, despite its simple definition and apparent ease of understanding, is a misleading metric.Its magnitude is easy to grasp, leading people to confidently make intuitive decisions based on its fluctuations. However, the reality is far more complex; AOV can show dramatic changes even when there’s no real underlying effect.
Conversely, significant changes can go unnoticed. A strong negative effect could be masked by just a few high-spending customers landing in a poorly performing variation. So, now you know: just as you do for conversion rates, rely on statistical tests for your AOV decisions.
Hello! I’m Léo, Senior Product Manager at AB Tasty. I’m in charge of AB Tasty’s JavaScript tag that is currently running on thousands of websites around the world. As you can guess, my roadmap is full of topics around data collection, privacy, and… performance.
It’s why I’m so excited to give an update on our performance, and how we’ve worked hard to be the best. We’re now providing loading times up to 4 faster than other solutions on the market.
In a world where every second counts, slow-loading pages are the fast track to lost revenue. At AB Tasty, we know that speed isn’t just about convenience; it’s about delivering the smooth, reliable experience that today’s consumers expect.
That’s why we’re thrilled to be recognized by ThirdPartyWeb.today for having one of the lowest impacts on web performance among top experimentation and personalization platforms. This acknowledgment affirms our commitment to speed, scalability, and brand satisfaction.
But what does this actually mean for brands using AB Tasty?
Let’s dive into how prioritizing performance can improve your SERP rankings, customer experience (CX), and overall campaign effectiveness.
Why Web Performance Impacts Your Bottom Line
Imagine clicking on a page that seems to take forever to load. Chances are, you’d be out of there faster than you could say “conversion rate.” And you wouldn’t be alone: slow page load times can lead to increased bounce rates, missed opportunities, and, ultimately, frustrated visitors.
Good performance translates into smoother customer journeys, which leads to better engagement and, most importantly, higher conversion rates.
ThirdPartyWeb.today: The Performance Benchmark
ThirdPartyWeb.today is an independent performance data visualization initiative that analyzes the impact of various platforms on page speed. It ranks tools according to their performance cost, drawing data from nearly 4 million websites to create an unbiased performance benchmark. For brands aiming to deliver a seamless experience without sacrificing speed, ThirdPartyWeb.today provides a reliable guide for evaluating the performance impact of their tools.
Being recognized as one of the most performance-friendly Experience Optimization platforms by ThirdPartyWeb.today means our clients know they’re partnering with a technology designed with speed in mind.
What Makes AB Tasty the Fastest?
Our tech teams have worked tirelessly to make AB Tasty not only an intuitive experimentation and personalization platform, but one that prioritizes high performance. Here’s a quick look at the innovations that make AB Tasty so fast and reliable:
Modular Architecture with Innovative Dynamic Importing and Smart Caching Technology Our platform is built with a modular architecture, where only essential code is loaded for each campaign. This keeps file sizes lean, reducing load time and resource consumption.Our proprietary smart caching technology ensures that visitors only need to load the data they haven’t accessed before. By minimizing redundant data calls, we significantly reduce load times across all devices. We also provide worldwide API endpoints and have a global CDN presence with multiple Edge locations and regional Edge caches for fast response times no matter where you and your site visitors are.
Performance Center AB Tasty’s dedicated Performance Center allows you to monitor your campaign performance in real-time. This tool gives you full transparency into what’s happening behind the scenes, so you can make adjustments as needed to keep things running smoothly. It provides recommendations to help you monitor and improve tag weight. Learn all about it here.
Single-Page Application (SPA) Compatibility AB Tasty’s platform is SPA-compatible without requiring custom code, making it easier for developers to integrate AB Tasty into their tech stack. AB Tasty is running on a native Vanilla TypeScript framework. Our tag is compatible with modern JS frameworks, including React, Angular, Vue, Meteor or Ember. The tag is unique for all environments and doesn’t require any additional implementation. Many of our customers have left their previous provider due to challenges with SPA pages. In these tools, changes are often not “sticky” or flicker when there is a dynamic content load. SPA tests in these environments often require custom code for each test, which makes testing more complicated and less user-friendly.
Flicker-Free Experiences AB Tasty’s tag uses a blended approach of both synchronous and asynchronous scripts to eliminate flicker, while maintaining optimised performance. Other solutions will prescribe “anti-flicker” snippets to eliminate flicker, which is not a recommended practice. It means hiding the body’s content while the tag loads, which ultimately delays the rendering of the site. This causes a worse user experience, increases your Largest Contentful Paint (LCP) metric, and may ultimately lead to increased bounce rates and decreased conversions. In contrast, AB Tasty’s synchronous tag uses 3kb of render-blocking to allow the tag to execute quickly before the page loads, as opposed to blocking the visibility of the page for the full package size.
And that translates to…
First loading time < 100ms Caching loading time < 10ms Execution time < 500 milliseconds Minimal Lighthouse Core Web Vitals impact
Cheers to Our Product and Tech Teams
This wouldn’t be possible without the dedication of our Product and Tech teams (thanks team!). We’ve dared to innovate, pushing the limits of what’s possible with web performance in the experimentation and personalization space.
The Bottom Line
When brands choose AB Tasty, they’re choosing a platform that prioritizes both innovation and performance. By minimizing impact on web performance, we’re helping brands deliver faster, better experiences that delight customers and drive results.
Curious to learn more about? Contact us today to discover what else sets us apart.
Building a culture of experimentation requires an appetite for iteration, a fearless approach to failure and a test-and-learn mindset. The 1000 Experiments Club podcast digs into all of that and more with some of the most influential voices in the industry.
From CEOs and Founders to CRO Managers and more, these experts share the lessons they’ve learned throughout their careers in experimentation at top tech companies and insights on where the optimization industry is heading.
Whether you’re an A/B testing novice or a seasoned pro, here are some of our favorite influencers in CRO and experimentation that you should follow:
Ronny Kohavi
Ronny Kohavi, a pioneer in the field of experimentation, brings over three decades of experience in machine learning, controlled experiments, AI, and personalization.
He was a Vice President and Technical Fellow at Airbnb. Prior to that, he was Technical Fellow and Corporate Vice President at Microsoft, where he led the analysis and experimentation team (ExP). Before that, he was Director of Personalization and Data Mining at Amazon.
Ronny teaches an online interactive course on Accelerating Innovation with A/B Testing, which was attended by over 800 students
Ronny’s work has helped lay the foundation for modern online experimentation, influencing how some of the world’s biggest companies approach testing and decision-making.
He advocates for a gradual rollout approach over the typical 50/50 split at launch:
“One thing that turns out to be really useful is to start with a small ramp-up. Even if you plan to go to 50% control and 50% treatment, start at 2%. If something egregious happens—like a metric dropping by 10% instead of the 0.5% you’re monitoring for—you can detect it in near real time.”
This slow ramp-up helps teams catch critical issues early and protect user experience.
Talia Wolf is a conversion optimization specialist and founder & CEO of Getuplift, where she helps businesses boost revenue, leads, engagement, and sales through emotional targeting, persuasive design, and behavioral data.
She began her career at a social media agency, where she was introduced to CRO, then served as Marketing Director at monday.com before launching her first agency, Conversioner, in 2013.
Talia teaches companies to optimize their online presence using emotionally-driven strategies. She emphasizes that copy and visuals should address customers’ needs rather than focusing solely on the product.
For Talia, emotional marketing is inherently customer-centric and research-based. From there, experiments can be built into A/B testing platforms using a clear North Star metric—whether checkouts, sign-ups, or add-to-carts—to validate hypotheses and drive growth.
Elissa Quinby is the Head of Product Marketing at e-commerce acceleration platform Pattern, with a career rooted in retail, marketing, and customer experience.
Before joining Pattern, she led retail marketing as Senior Director at Quantum Metric. She began her career as an Assistant Buyer at American Eagle Outfitters, then spent two years at Google as a Digital Marketing Strategist. Elissa went on to spend eight years at Amazon, holding roles across marketing, program management, and product.
Elissa emphasizes the importance of starting small to build trust with new customers. “The goal is to offer value in exchange for data,” she explains, pointing to first-party data as the “secret sauce” behind many successful companies.
She encourages brands to experiment with creative ways of gathering customer information—always with trust at the center—so they can personalize experiences and deepen customer understanding over time.
Lukas Vermeer, Director of Experimentation at Vista, is an expert in designing, implementing, and scaling experimentation programs. He previously spent over eight years at Booking.com, where he held roles as a product manager, data scientist, and ultimately Director of Experimentation.
With a background in machine learning and AI, Lukas specializes in building the infrastructure and processes needed to scale testing and drive business growth. He also consults with companies to help them launch and accelerate their experimentation efforts.
Given today’s fast-changing environment, Lukas believes that roadmaps should be treated as flexible guides rather than rigid plans: “I think roadmaps aren’t necessarily bad, but they should acknowledge the fact that there is uncertainty. The deliverable should be clarifications of that uncertainty, rather than saying, ‘In two months, we’ll deliver feature XYZ.’”
Instead of promising final outcomes, Lukas emphasizes embracing uncertainty to make better, data-informed decisions.
Jonny Longden is the Chief Growth Officer at Speero, with over 17 years of experience improving websites through data and experimentation. He previously held senior roles at Boohoo Group, Journey Further, Sky, and Visa, where he led teams across experimentation, analytics, and digital product.
Jonny believes that smaller companies and startups—especially in their early, exploratory stages—stand to benefit the most from experimentation. Without testing, he argues, most ideas are unlikely to succeed.
“Without experimentation, your ideas are probably not going to work,” Jonny says. “The things that seem obvious often don’t deliver results, and the ideas that seem unlikely or even a bit silly can sometimes have the biggest impact.”
For Jonny, experimentation isn’t just a tactic—it’s the only reliable way to uncover what truly works and drive meaningful, data-backed progress.
Ruben de Boer is a Lead CRO Manager at Online Dialogue and founder of Conversion Ideas, with over 14 years of experience in data and optimization.
At Online Dialogue, he leads the team of Conversion Managers—developing skills, maintaining quality, and setting strategy and goals. Through his company, Conversion Ideas, Ruben helps people launch their careers in CRO and experimentation by offering accessible, high-quality courses and resources.
Ruben believes experimentation shouldn’t be judged solely by outcomes. “Roughly 25% of A/B tests result in a winner, meaning 75% of what’s built doesn’t get released—and that can feel like failure if you’re only focused on output,” he explains.
Instead, he urges teams to shift their focus to customer-centric insights. When the goal becomes understanding the user—not just releasing features—the entire purpose of experimentation evolves.
David Mannheim is a digital experience strategist with over 15 years of expertise helping brands like ASOS, Sports Direct, and Boots elevate their conversion strategies.
He is the CEO and founder of Made With Intent, focused on advancing innovative approaches to personalization through AI. Previously, he founded User Conversion, which became one of the UK’s largest independent CRO consultancies.
David recently authored a book exploring what he calls the missing element in modern personalization: the person. “Remember the first three syllables of personalization,” he says. “That often gets lost in data.”
He advocates for shifting focus from short-term gains to long-term customer value—emphasizing metrics like satisfaction, loyalty, and lifetime value over volume-based wins.
“More quality than quantity,” David explains, “and more recognition of the intangibles—not just the tangibles—puts brands in a much better place.”
Marianne Stjernvall has over a decade of experience in CRO and experimentation, having executed more than 500 A/B tests and helped over 30 organizations grow their testing programs.
Marianne is the founder of Queen of CRO and co-founder of ConversionHub, Sweden’s most senior CRO agency. As an established CRO consultant, she helps organizations build experimentation-led cultures grounded in data and continuous learning.
Marianne also teaches regularly, sharing her expertise on the full spectrum of CRO, A/B testing, and experimentation execution.
She stresses the importance of a centralized testing approach:
“If each department runs experiments in isolation, you risk making decisions based on three different data sets, since teams will be analyzing different types of data. Having clear ownership and a unified framework ensures the organization works cohesively with tests.”
Ben Labay is the CEO of Speero, blending academic rigor in statistics with deep expertise in customer experience and UX.
Holding degrees in Evolutionary Behavior and Conservation Research Science, Ben began his career as a staff researcher at the University of Texas, specializing in data modeling and research.
This foundation informs his work at Speero, where he helps organizations leverage customer data to make better decisions.
Ben emphasizes that insights should lead to action and reveal meaningful patterns. “Every agency and in-house team collects data and tests based on insights, but you can’t stop there.”
Passionate about advancing experimentation, Ben focuses on developing new models, applying game theory, and embracing bold innovation to uncover bigger, disruptive insights.
André Morys, CEO and founder of konversionsKRAFT, has nearly three decades of experience in experimentation, digital growth, and e-commerce optimization.
Fueled by a deep fascination with user and customer experience, André guides clients through the experimentation process using a blend of data, behavioral economics, consumer psychology, and qualitative research.
He believes the most valuable insights lie beneath the surface. “Most people underestimate the value of experimentation because of the factors that are hard to measure,” André explains.
“You cannot measure the influence of experimentation on your company’s culture, yet that impact may be ten times more important than the immediate uplift you create.”
This philosophy is central to his “digital experimentation framework,” which features his signature “Iceberg Model” to capture both measurable and intangible effects of testing.
Jeremy Epperson is the founder of Thetamark and has dedicated 14 years to conversion rate optimization and startup growth. He has worked with some of the fastest-growing unicorn startups in the world, researching, building, and implementing CRO programs for more than 150 growth-stage companies.
By gathering insights from diverse businesses, Jeremy has developed a data-driven approach to identify testing roadblocks, allowing him to optimize CRO processes and avoid the steep learning curves often associated with new launches.
In his interview, Jeremy emphasizes focusing on customer experience to drive growth. He explains, “We will do better as a business when we give the customer a better experience, make their life easier, simplify conversion, and eliminate the roadblocks that frustrate them and cause abandonment.”
His ultimate goal with experimentation is to create a seamless process from start to finish.
Chad Sanderson is the CEO and founder of Gable, a B2B data infrastructure SaaS company, and a renowned expert in digital experimentation and large-scale analysis.
He is also a product manager, public speaker, and writer who has lectured on topics such as the statistics of digital experimentation, advanced analysis techniques, and small-scale testing for small businesses.
Chad previously served as Senior Program Manager for Microsoft’s AI platform and was the Personalization Manager for Subway’s experimentation team.
He advises distinguishing between front-end (client-side) and back-end metrics before running experiments. Client-side metrics, such as revenue per transaction, are easier to track but may narrow focus to revenue growth alone.
“One set of metrics businesses mess up is relying only on client-side metrics like revenue per purchase,” Chad explains. “While revenue is important, focusing solely on it can drive decisions that overlook the overall impact of a feature.”
With a background in Global Business Management and Marketing, Computer Science, and Industrial Engineering, Carlos founded Floqq—Latin America’s largest online education marketplace.
In 2014, he founded Product School, now the global leader in Product Management training.
Carlos believes experimentation has become more accessible and essential for product managers. “You no longer need a background in data science or engineering to be effective,” he says.
He views product managers as central figures at the intersection of business, design, engineering, customer success, data, and sales. Success in this role requires skills in experimentation, roadmapping, data analysis, and prototyping—making experimentation a core competency in today’s product landscape.
Bhavik Patel is the Data Director at Huel, an AB Tasty customer, and the founder of CRAP Talks, a meetup series connecting CRO professionals across Conversion Rate, Analytics, and Product.
Previously, he served as Product Analytics & Experimentation Director at Lean Convert, where he led testing and optimization strategies for top brands. With deep expertise in personalization, experimentation, and data-driven decision-making, Bhavik helps teams evolve from basic A/B testing to strategic, high-impact programs.
With a focus on experimentation, personalization, and data-driven strategy, Bhavik leads teams in creating better digital experiences and smarter testing programs.
His philosophy centers on disruptive testing—bold experiments aimed at breaking past local maximums to deliver statistically meaningful results. “Once you’ve nailed the fundamentals, it’s time to make bigger bets,” he says.
Bhavik also stresses the importance of identifying the right problem before jumping to solutions: “The best solution for the wrong problem isn’t going to have any impact.”
Rand Fishkin is the co-founder and CEO of SparkToro, creators of audience research software designed to make audience insights accessible to all.
He also founded Moz and co-founded Inbound.org with Dharmesh Shah, which was later acquired by HubSpot in 2014. Rand is a frequent global keynote speaker on marketing and entrepreneurship, dedicated to helping people improve their marketing efforts.
Rand highlights the untapped potential in niche markets: “Many founders don’t consider the power of serving a small, focused group of people—maybe only a few thousand—who truly need their product. If you make it for them, they’ll love it. There’s tremendous opportunity there.”
A strong advocate for risk-taking and experimentation, Rand encourages marketers to identify where their audiences are and engage them directly there.
Shiva Manjunath is the Senior Web Product Manager of CRO at Motive and host of the podcast From A to B. With experience at companies like Gartner, Norwegian Cruise Line, and Edible, he’s spent years digging into user behavior and driving real results through experimentation.
Shiva is known for challenging the myth of “best practices,” emphasizing that optimization requires context, not checklists. “If what you believe is this best practice checklist nonsense, all CRO is just a checklist of tasks to do on your site. And that’s so incorrect,” he says.
At Gartner, a simplified form (typically seen as a CRO win) led to a drop in conversions, reinforcing his belief that true experimentation is about understanding why users act, not just what they do.
Through his work and podcast, Shiva aims to demystify CRO and encourage practitioners to think deeper, test smarter, and never stop asking questions.
Running hundreds of experiments each year is a sign of a mature, data-driven organization – but it also comes with challenges.
How do you ensure that every test is running smoothly, and that critical issues don’t slip through the cracks?
At AB Tasty, we’ve listened to our clients’ pain points and are excited to announce the launch of Experiment Health Check: a new feature designed to make experimentation safer, smarter, and more efficient.
The Challenge: Keeping Experiments Healthy at Scale
For leading brands running over 100 campaigns a year, experimentation is at the heart of digital optimization.
But with so many campaigns running simultaneously, manually checking reports every day to spot issues is time-consuming and inefficient. Worse, problems like underperforming variations or sample ratio mismatches (SRM) can go unnoticed, leading to lost revenue or inconclusive results.
Our Solution: Experiment Health Check
Experiment Health Check is an automated monitoring system built directly into AB Tasty. It proactively alerts you to issues in your experiments, so you can act fast and keep your testing program on track.
Key Features:
Automated Alerts: Get notified in-product (and by email, if you choose) when an experiment encounters a critical issue, such as:
Centralized Dashboard: Super-admins can view all alerts across accounts for a global overview.
Customizable Notifications: Choose which alerts to display and how you want to receive them.
Why It Matters
Proactive, Not Reactive: No more waiting until the end of a test or sifting through reports to find problems. Experiment Health Check surfaces issues as soon as they’re detected.
Saves Time: Focus on insights and strategy, not manual monitoring.
Peace of Mind: Most clients will rarely see alerts – only about 2% of campaigns encounter SRM issues – so you can be confident your experiments are running smoothly.
What’s Next?
Experiment Health Check is available to all AB Tasty clients as of June 2025.
Simply activate it in your dashboard to start benefiting from automated experiment monitoring. We’re committed to evolving this feature with more alert types and integrations based on your feedback.
