Debugging Server-Side Experimentation Faster with Live Hits
Emily Healy
When teams run server-side experiments, one of the biggest challenges is validating that everything is working correctly before and after launch.
Unlike client-side experimentation, where visual checks can often help confirm a setup, server-side experimentation depends heavily on event flows, payload quality, and implementation accuracy. If something is misconfigured, teams may not notice immediately. In many cases, they have to wait for reporting to refresh before they can confirm whether data is being collected as expected.
That delay can slow down QA, make troubleshooting harder, and reduce confidence at launch.
The server-side debugging challenge
For product, engineering, and experimentation teams, implementation validation is a critical part of the workflow. Before a campaign goes live, they often need to answer a few simple but important questions:
Are hits actually reaching the platform?
Are the right events being sent?
Do the payload details match what was expected?
Is everything working properly in production after launch?
Without real-time visibility, answering those questions can take longer than it should. Teams may need to wait for aggregated reporting or rely on manual checks across multiple tools. That creates friction in QA cycles and can make debugging more complex, especially in fast-moving release environments.
Introducing Live Hits
Live Hits is designed to make server-side QA and debugging much easier.
It provides a real-time stream of SDK events as they reach the platform, allowing teams to validate implementation immediately instead of waiting for reporting updates. This gives users direct visibility into what is being sent, helping them troubleshoot faster and launch with more confidence.
Rather than working from delayed, aggregated data, teams can inspect incoming hits as they happen.
What Live Hits helps teams do
Live Hits is especially useful during two key moments:
1. During QA before launch
When a campaign or feature is ready for validation, teams can use Live Hits to confirm that the expected events are arriving correctly. This helps verify that implementation is complete and that the right information is being sent.
2. Right after launch in production
Once a campaign is live, teams can run a second check to confirm that traffic is flowing as expected in the real environment. This helps catch issues early and adds an extra layer of confidence at go-live.
Why this matters
Real-time visibility can make a major difference for teams working on server-side experimentation.
Key benefits include:
Faster debugging
Identify issues without waiting for reporting refreshes
Smoother QA workflows
Validate implementation before launch
Better troubleshooting
Inspect detailed event information when something looks off
For teams running complex experimentation programs, these advantages can reduce back-and-forth between product, engineering, and QA while speeding up time to validation.
A more practical way to validate implementation
One of the most useful aspects of Live Hits is that it helps teams move from assumption to confirmation.
Instead of asking, “Did the event fire?” and waiting for reports, users can quickly verify:
the type of hit received
the associated identifiers
the event details being transmitted
whether the payload matches expectations
This makes it easier to investigate implementation issues, validate tracking logic, and confirm that a campaign is ready to move forward.
Built for real experimentation workflows
In practice, server-side experimentation often requires close collaboration across multiple teams. Product managers want confidence in setup, developers want to confirm implementation, and QA teams need a reliable way to validate behavior before launch.
Live Hits supports that workflow by giving teams a shared, immediate view of incoming SDK activity. It helps simplify the path from implementation to launch, especially when speed and accuracy both matter.
Why real-time validation is becoming essential
As experimentation programs mature, teams need more than reporting alone. They need tools that help them validate faster, troubleshoot earlier, and reduce uncertainty during rollout.
That is exactly where Live Hits adds value.
By giving teams real-time visibility into server-side events, it helps turn debugging and QA into a faster, more reliable process. For organizations looking to scale experimentation with confidence, that kind of visibility can be a meaningful operational advantage.
Final thoughts
Server-side experimentation offers flexibility and control, but it also raises the bar for implementation validation. Waiting for aggregated reports is not always enough when teams need to debug quickly and launch confidently.
Live Hits from AB Tasty helps close that gap by making server-side event validation immediate, practical, and easier to act on.
If your teams are looking for a better way to QA server-side campaigns and verify implementation in real time, Live Hits is built for exactly that.
The Hidden Cost of Ignoring Your E-Commerce KPIs (And How to Fix It)
Emily Healy
For e-commerce teams, the pressure to deliver results is constant. Whether you’re a merchandiser, a buyer, or a digital leader, you’re expected to show how your work translates into business value. But in the day-to-day rush, it’s easy to lose sight of the numbers that matter most.
The Real Challenge: Out of Sight, Out of Mind
Most e-commerce platforms offer a wealth of data. There are dashboards, reports, and analytics tools that can tell you almost anything about your site’s performance. But here’s the problem: these insights are often buried, hard to access, or only reviewed during quarterly business reviews.
As a result:
After the initial setup, strategies can become “set and forget.”
Opportunities for improvement are missed because no one is regularly looking at the right KPIs.
When it’s time to prove ROI, it’s a scramble to pull together evidence and justify decisions.
When performance data isn’t front and center, it’s easy for teams to become reactive instead of proactive. The focus shifts from continuous improvement to simply keeping things running.
A Better Way: Make KPIs Part of the Weekly Routine
What if, instead of waiting for high-stakes quarterly reviews, you had a simple, structured way to check in every week? A routine that keeps your KPIs visible, your strategies healthy, and your team focused on improvement – not just maintenance.
That’s the thinking behind AB Tasty’s Performance Digest: a weekly loop that connects three things into one flow:
1
Step 1
Weekly Digest Email
A weekly email that highlights key performance signals and brings you back into the platform.
2
Step 2
Reporting on Business Impact
Stay up to date with KPIs designed to be buyer-friendly.
3
Step 3
Guided ideas for improvement
Think of our AI as a coach that helps you prioritize what to optimize next and gives step-by-step instructions.
This isn’t about adding more to your plate. It’s about making performance a habit, not a hurdle.
A Weekly Habit That Tells a Clear ROI Story
One of the hardest parts of e-commerce reporting isn’t collecting data — it’s telling a clear story about business impact.
AB Tasty’s Performance Digest makes that story easier by emphasizing direct contribution: the business generated when a shopper clicks a recommended or merchandised item and purchases within the same session.
Just as importantly, it builds a weekly routine around that story. Instead of hoping teams remember to log in (or only revisiting performance during quarterly reviews), Performance Digest starts in your inbox with a weekly snapshot of what moved — so KPIs stay visible and decision-making stays consistent.
From “What Happened?” to “What to Do Next”
Dashboards can show you what happened, but the real bottleneck is deciding what to do next. That’s why Performance Digest will include a small set of guided improvement ideas to help you prioritize where to invest time, spot opportunities you might miss in a dashboard-only workflow, and iterate with confidence.
This weekly loop also makes ROI easier to prove. Quarterly reviews often create high-stakes, retroactive conversations. A weekly cadence creates a steady trail of decisions and outcomes — what changed, what improved, and why
Takeaway: Make Performance a Habit, Not a Hurdle
The most successful e-commerce teams don’t wait for problems to show up in quarterly reports. They build habits that keep performance and improvement part of the weekly workflow. The key is to make your most important KPIs visible, actionable, and impossible to ignore.
Because when ROI is always top of mind, better results tend to follow.
FAQs about E-commerce KPIs
How often should e-commerce teams review KPIs?
Weekly KPI reviews are often the most effective cadence for e-commerce teams because they’re frequent enough to catch issues early, but light enough to sustain as a habit.
How do you prove ROI from merchandising and product recommendations?
To prove ROI, connect merchandising actions to business impact using metrics that tie clicks and placements to outcomes (like revenue, conversion, or contribution metrics tied to on-site interactions).
With AB Tasty’s Performance Digest, you create a weekly “paper trail” of performance signals and actions taken, making ROI conversations easier and less reactive at quarter-end.
How can I keep e-commerce KPIs visible without living in dashboards?
AB Tasty’s Performance Digest brings key performance signals into a weekly email, so e-commerce teams stay on top of their results.
How do we turn KPI reporting for e-commerce into clear next steps (not just charts)?
AB Tasty’s Performance Digest includes guided ideas for improvement, helping you prioritize what to optimize next instead of staring at dashboards and guessing.
What are e-commerce KPIs and why do they matter?
E-commerce KPIs are measurable metrics – like conversion rate, revenue, and average order value – that show how your site and merchandising efforts impact business results.
Conversion is a conversation: how GenAI is reshaping e-commerce and travel
Emily Healy
Generative AI is now creating a massive shift in how online buyers discover and compare products. That’s why we’ve put together a roadmap to help you understand the 2026 consumer in our e-book, The Spontaneous Shift: Consumer E-Commerce Trends for 2025. For e-commerce and travel brands this represents a real paradigm shift in how digital marketing works; one they will need to adapt to, and fast, if they want to survive.
Meet your customer’s new shopping assistant
While Google search is still the dominant force in discovery, our research shows that other channels are rapidly gaining ground. And the fastest growing channel of all is Generative AI (like ChatGPT, Google’s Gemini, or Claude from Anthropic). Use of Gen AI tools by online shoppers in the discovery phase has grown 75% in just the last year, rising from 8% in 2025 to 18% as it quickly moves from a novelty to a utility.
But while more people are choosing to use AI, the level of uptake differs across generations. Perhaps not surprisingly, younger shoppers are more comfortable using AI tools to find what they need. 32% of Gen Z now say that they find AI helpful in their online buying journey. Millennials are not far behind, with 30% of them seeing AI as a helpful assistant, but this number falls to 13% of Baby Boomers.
Question everything about search
What’s clear is that the online behavior of consumers is changing and at pace. Younger generations especially are no longer content with scrolling through a list of blue links. Instead, they’re asking questions – and expecting answers. They aren’t just typing keywords like “red sneakers” into their browser; they’re giving AI prompts, like “Find me a pair of red sneakers that look cool and are under $80.”
This is a profound shift and one that has big implications for e-commerce and travel brands. Firstly, it means consumers no longer need to visit multiple websites to compare different products and services. They can do all of that and more by simply asking questions to an AI interface: “Is this sustainable?” “Will this work for my specific use case?” “How does this compare to their competitor’s offer?” The transaction is quickly becoming a conversation.
What this signals is a move away from carefully crafted product detail pages (PDPs) and towards much more dynamic interactions. If your brand can’t answer these questions in the conversational spaces where they’re being asked, either an on-site AI chatbot or a third-party platform, you’re effectively invisible. You don’t exist in an online consumer’s decision-making loop.
The new frontier of optimization
That in turn presents brands with a unique challenge. When online buyers perform a Google search or search directly on your website, you can see their query. But when they ask a question to a third-party AI platform, that happens inside a metaphorical “black box”. You can’t see what question they asked, and you can’t (currently) buy an ad to influence it.
This “invisible intent” requires a new kind of optimization. Because you’re no longer just writing for humans or typical SEO algorithms; you’re also writing for the Large Language Models (LLMs) of Generative AI. Fortunately, the core SEO principles of E-E-A-T (Experience, Expertise, Authoritativeness, Trustworthiness) can still provide brands with a competitive edge.
This is because LLMs are designed to recommend what they interpret as being the most credible answer, not just the one with the most (or best) keywords. If you want GenAI to recommend your products or services, you’ll have to give it proof that your brand is the authority.
To stay visible in this conversational future, that means your content strategy needs to move from volume to value. And if LLMs are to place any trust in your user reviews, they’ll need to be authentic. You also need to ensure that product data is structured with schema markup so it can be “read” by these new gatekeepers. It needs to be rich enough to answer the nuanced questions of online shoppers, not just fill a spec sheet.
Changing the face of travel
Human interaction has always been at the heart of the travel industry. And travelers have traditionally been happy to receive personalized recommendations or resolve any issues they might have by speaking with a real person. But this change in online buyer behavior from keyword search to asking questions of GenAI means the travel sector is perhaps even more exposed than most to this disruption.
Our research shows that 36% of travelers have now used AI-powered tools, chatbots, or virtual assistants to book travel or solve issues and found them helpful. Another 32% haven’t tried using these yet but are open to doing so. Again, Gen Z are the most open to AI, with 49% saying they have used AI tools and found them helpful when booking travel. And while this figure is 41% for Millennials, it falls to 21% of Baby Boomers.
Focus on quality
This is good news for travel brands looking to embrace on-site AI chatbots to help visitors plan travel. But at the same time, there’s also nothing to stop those same online buyers from doing that on a third-party platform. A prompt like “Plan a 5-day family-friendly trip to Lisbon in May, staying near the city center with a budget of $2000,” can generate a complete itinerary with flight suggestions, hotel options, and activity booking links. All without visitors ever going near a brand’s website.
That means there’s a real possibility for travel brands of being reduced to a single item in an AI-generated plan. And if they lose the ability to showcase a unique brand experience on their website, travel companies like hotels and airlines risk being compared merely on price and basic features alone. Loyalty programs and creating exceptional customer experiences are likely to be more important than ever.
Travel is also time sensitive. A third-party AI platform is unlikely to recommend a hotel or a flight if it can’t access real-time availability, accurate pricing, and clear policies. This requires travel companies to be laser-focused on the quality and accessibility of their data. Again, the best way to ensure that GenAI recommends your brand to online buyers is to have rich, well-structured information that is instantly available.
Conclusion
The world of e-commerce and online travel is undergoing a real paradigm shift in how online buyers discover and compare products. And the rules are still changing in real-time. Ensuring that your data is as authoritative as possible and easily accessible to GenAI is currently your best bet to remain visible in this new environment. The only viable way to do this is to build a culture of continuous experimentation; testing, learning, and iterating towards your “better”.
Takeaways for e-commerce and travel brands
Test your GenAI visibility: Go to ChatGPT or Gemini and ask questions about your product category. If your brand doesn’t show up, you have an optimization gap to fill.
Develop AI-friendly product data structure: Implementing comprehensive schema markup across your site is no longer optional, it’s the price of entry.
Create AI-friendly content: Develop clear, factual, easily digestible content that directly answers questions your customers might have. This makes it much more likely that it will be used by an LLM.
Track GenAI adoption monthly: Monitor the amount of traffic to your website coming from third-party Gen AI tools and how this affects sales.
Real-Time Personalization in 2026: How to Meet Customer Expectations
Emily Healy
Personalization has long been a goal for brands looking to create more relevant and engaging experiences. Traditionally, this has meant using static rules and segments – showing different content to new visitors, returning customers, or VIP shoppers.
While this approach has value, it often falls short when customer needs and interests change quickly.
Why Real-Time Personalization Matters More Than Ever
Today’s customers are unpredictable. They jump between devices, change their minds mid-session, and expect brands to keep up. It’s very common for shoppers to change their intent during a single visit.
For example: a user who starts out searching for a small purse for everyday fashion [category: bag] might begin looking for black sunglasses for an upcoming beach trip [category: accessories] – all in one session. Real-time personalization adapts to user needs in the moment.
Traditional personalization is based on fixed segments and rules.
It can’t keep up with rapid shifts in customer needs and preferences. Traditional personalization treats users the same way throughout their visit, missing crucial signals and opportunities to engage.
The Challenge: Static Personalization’s Limits
Most brands have invested in standard personalization. It’s a great foundation, using segments like “new visitor” or “VIP” to tailor experiences. But static rules are slow to react. They’re built on who a customer was, not who they are right now or how they are behaving.
The result? Missed chances to upsell, cross-sell, or simply delight your customers with relevant content. In a world where attention spans are short and competition is fierce, that’s a risk brands can’t afford.
The Solution: Real-Time Personalization with AdaptiveCX
Real-time personalization is about responding to customer intent as it happens. Instead of relying solely on historical data or static segments, real-time solutions monitor user actions (clicks, searches, time spent on pages) and adjust the experience instantly.
So, how can brands personalize in real time in 2026? The answer lies in solutions like AdaptiveCX– technology designed to sense, interpret, and respond to customer intent as it happens. Adopting real-time personalization doesn’t mean abandoning what works. Think of it as evolving your strategy – layering real-time capabilities on top of your existing efforts.
Here’s what real-time personalization looks like:
Continuous monitoring: Every interaction is tracked as it happens.
Intent recognition: The system identifies when a user’s interests or goals change, even within a single session.
Instant updates: Content, offers, and recommendations are refreshed on the fly to match the user’s current intent.
Example: Let’s say there’s a user who has a typically of using one brand to rent a car for business travels and now this user wants to use the same brand to explore family vacation rental options. With real-time personalization, the website immediately shifts to highlight family-friendly deals and relevant upsells, ensuring the experience stays relevant throughout the session.
How to Personalize in Real Time in 2026
Use behavioral data: Go beyond static segments by analyzing real-time actions – what users are clicking, searching, and viewing right now.
Integrate across channels: Ensure your personalization engine works seamlessly on your website, mobile app, email, and even in-store.
Prioritize intent: Focus on what users are doing in the moment, not just who they are or what they did in the past.
Test and refine: Continuously experiment with different approaches to see what drives the best results.
Respect privacy: Be transparent about how you use data and give users control over their personalization preferences.
The Impact of Real-Time Personalization
Brands that implement real-time personalization see clear benefits:
Higher engagement and conversion rates
Increased customer satisfaction and loyalty
More effective upselling and cross-selling
A stronger competitive position
Moving Forward
Personalization in 2026 is about more than just knowing your customers. It’s about understanding and responding to what they want in the moment. Real-time personalization helps brands stay relevant, capture more opportunities, and deliver experiences that truly resonate.
We’re continuously improving Feature Experimentation & Rollouts (FE&R) to make your experience smoother, faster, and more reliable. Here are the latest updates now available:
Data Explorer: Now available in FE&R
The Data Explorer, already available for Web Experimentation, is now accessible in Feature Experimentation & Rollouts. You can now explore and export your server-side data directly from the platform, without relying on external or technical tools like Postman.
Access and export raw hits (SDK events, API calls)
Apply filters and dimensions (campaign, environment, feature, user properties, time range, etc.)
Define custom date ranges and result limits
Generate API payloads directly from the UI
This feature was one of the most requested improvements by our clients. It increases transparency, improves trust in data collection, and brings stronger consistency between client-side and server-side capabilities within One Platform.
When implementing or debugging a campaign, waiting for aggregated reports can slow you down. With Live Hits, you can now visualize incoming SDK events in real time and instantly verify that your implementation is working as expected.
No more guesswork. No more waiting.
Live Hits significantly improves QA workflows and reduces implementation friction for technical teams.
You can quickly confirm:
That events are properly triggered
That traffic is correctly allocated
That variations are being served as expected
👉 Access in Reporting → Button next to the filter.
Server-Side Reporting: Evolution
AND / OR filter operators
You can now switch between AND / OR operators for multiple values within the same filter type.
This gives you much more flexibility when segmenting your data and running deeper analysis directly inside your reports.
More precise segmentation means more accurate insights.
RevenueIQ*: now available for Server-Side 💰
*RevenueIQ is AB Tasty’s patented statistical engine that transforms experiment results into clear, reliable revenue projections — helping teams move from CRO to true Revenue Optimization by confidently quantifying uplift and ROI for every variation.
RevenueIQ, already available and widely adopted on client-side experimentation, is now live for server-side reports.
You can now project the financial impact of your experiments directly from your server-side reporting.
Available in any report with a transaction goal under the “Revenue stats” tab, RevenueIQ provides:
Uplift per visitor: estimated revenue increase per visitor once deployed
Uplift per month: projected monthly revenue impact
Confidence intervals: lower, median, and upper revenue scenarios
Revenue chances to win: probability that the variation will increase revenue
This means clearer decisions, fewer inconclusive experiments, and a direct view of real business impact.
You can confidently quantify and share the ROI of every server-side experiment.
Conclusion
These latest enhancements to Feature Experimentation & Rollouts are designed to empower your teams with greater transparency, flexibility, and actionable insights. With real-time data access, advanced reporting options, and robust revenue projections, you can confidently optimize your experiments and drive measurable business impact.
KPIs, Custom Metrics, and Evi: Supercharge AI Insights in AB Tasty
Emily Healy
Our platform is above all designed to help you drive your business forward. That’s why in AB Tasty you can set up a range of metrics straight out of the box that have meaningful business impact. And our agentic AI, Evi, will help you analyze test results fast and provide you with valuable insights your team might otherwise miss.
We put the key in KPI
AB Tasty provides comprehensive reporting tools to evaluate the performance of your campaigns, focusing on the achievement of specific KPIs, called goals. These can be set up both at the account level and campaign level right out of the box. There are six different types of goals that you can chose to define as a KPI. These are:
Action tracking: Clicks, dwell time, iframe clicks, element visibility, scroll rate.
Page tracking: Visits to a particular page or group of pages.
Transactions: Average order value (AOV), purchase rate, total revenue.
Datalayer Goals: Tracking based on variables in your website’s data layer
Custom tracking: Trackers you’ve created with custom code on your website.
You shouldn’t get bogged down trying to define too many KPIs. And less is definitely more in this case. Try to identify 3 – 5 KPIs for each major business objective. If you Start with KPIs that are directly tied to business outcomes, they’ll help drive your business forward.
You measure you with custom metrics
These out of the box metrics are a great starting point. But no two businesses are the same, and your customer journey is unique. Standard KPIs might not capture interactions that are critical for you. By measuring what makes your business unique, you get insights that your competitors don’t have access to.
In AB Tasty, you can create Custom Trackings that are directly linked to your website’s DataLayer. This lets you create personalized metrics based entirely on your unique data. You can also create custom trackers using JavaScript, giving you the flexibility to measure almost any interaction on your website. These custom tracking events can even be used to replicate goals from other platforms, like Google’s GA4.
Real-time reporting happens now
We know that speed is critical to your business. For optimization to be effective, the feedback loop between action and insights needs to be as short as possible. Real-time reporting empowers your team to spot problems or winners within minutes, not days, enabling them to make confident decisions quickly.
That’s why AB Tasty offers real-time reporting capabilities that activate automatically when you push a campaign live. During the most critical initial phase of a test (the first 1,000 unique visitors or the first 12 hours), data for each goal and variation is updated every five minutes. We also provide a Live Hits monitoring tool which allows you to track event data at any time. This allows you to make faster, smarter decisions based on up-to-the-minute data.
Test with Confidence
Understanding the story your statistics are telling you is obviously crucial to making the right decisions. At AB Tasty, we use a Bayesian statistical model that provides you with intuitive, actionable results you can understand. We use two key figures to help you make more confident decisions:
A 95% confidence interval: This gives you a likely range for the true value of a gain. If the confidence interval for the gain is [2% 8%], we are 95% confident that the true uplift from this variation is between 2% and 8%. The remaining 5% represents the margin of error.
The chance to win: This is a direct probability that tells you how likely it is that a variation is better than the original. A 98% chance to win means there’s a 98% probability that the variation is the true winner.
Move at the speed of evidence with Evi
Having data and reporting from your tests is one thing, but analyzing those is another. Until recently, this has often been a time-consuming process and sometimes involved a little guesswork. But by integrating agentic AI into the reporting process, you can analyze data fast and receive valuable insights that your team might otherwise miss.
Evi is AB Tasty’s AI-powered marketing agent designed for evidence-based decision making. It transforms your data into clear, actionable strategies for repeatable, measurable results, ensuring every step you take is grounded in evidence.
With Evi, your team can:
Greatly accelerate the reporting process, enabling you to analyze campaign data within a matter of clicks not hours.
Extract deeper insights, all driven by actual website data using built-in AI analysis.
Evi features two separate AI agents accessible from the reporting page for each campaign in AB Tasty: Evi Analysis and Evi Explore.
Evi Analysis
Tired of spending hours sifting through data tables and colorful charts and wondering what they all mean? Evi Analysis will analyze your campaign data and deliver clear, actionable insights. It highlights winning variations and breaks down why they drive transactions so you can feel confident in your next move.
Simply enter your questions and Evi Analysis will process the underlying metrics, statistical significance, and objective performance to deliver clear, concise answers backed by campaign data. All in a matter of clicks. Use case examples might include:
Explaining the winning variation.
Challenging your hypothesis.
Giving you the best CRO practices based on your campaign results.
Evi Explore
Want to know if your tests will actually drive revenue? Evi Explore, powered by our own patented metric, RevenueIQ, makes it easy to interpret the results of campaigns that use a transactional goal (i.e. a goal that tracks purchases or revenue).
Evi Explore gives you a clear, statistically sound view of the revenue impact of each test variation before you launch. Rather than simply relying on traditional metrics like conversion rate or average order value (AOV), RevenueIQ combines these into a single metric to give you a direct view of revenue per visitor and per month.
This means no more inconclusive campaigns, no more ‘conversion rate vs AOV’ dilemmas, and a significant reduction in ‘undecidable’ tests. Teams can now project the revenue impact of a campaign before full rollout with confidence intervals for best- and worst-case scenarios. This gives you the confidence to make faster, more profitable decisions. And because none of our competitors currently offer a comparable metric, by using AB Tasty you receive insights others won’t have.
From Messy to Manageable: Organizing Experiments with Folders & Buckets
Emily Healy
If you’ve ever opened your company’s experimentation dashboard and felt overwhelmed by the sheer number of campaigns, you’re not alone. As businesses grow, so do the number of teams, projects, and experiments running at any given time. Suddenly, what started as a handful of tests can turn into a maze of overlapping campaigns, making it tough to find what you need.
That’s where Folders & Buckets come in. These two simple features can make a world of difference in how you manage, secure, and scale your experimentation efforts. Here’s how they work, why they matter, and some tips for getting the most out of them.
Too Many Experiments, Not Enough Organization
Picture this: your marketing, product, and development teams are all running their own experiments. Maybe you’ve got a few hundred campaigns live, or maybe it’s closer to a thousand. Either way, it’s easy for things to get messy. Important tests get buried, people accidentally edit the wrong campaign, and sometimes experiments even overlap – skewing your results or causing confusion.
This isn’t just a headache for your data team. It can slow down your whole organization and make it harder to get clear, actionable insights.
Folders: Your Experiment Filing Cabinet
Folders are exactly what they sound like: a way to group and organize your experiments in a way that makes sense for your business. But they’re much more than just a visual aid – they’re a powerful tool for access control and workflow management.
How Folders Work
Custom Organization: Structure folders by team (e.g., Marketing, Product), by project or sprint, by product line, or even by page type (e.g., Homepage, Checkout). The choice is yours.
Granular Permissions: Assign users to specific folders with different roles – viewer, editor, or admin. By default, new users see nothing until they’re granted access, minimizing risk and keeping sensitive experiments secure.
Flexible Access: Users can be given access to multiple folders, with different roles in each. This is perfect for organizations where people wear multiple hats or collaborate across teams.
Why Folders Matter
Clarity: Users see only the experiments relevant to them, reducing clutter and confusion.
Security: Sensitive or high-impact experiments are visible only to authorized users.
Agility: As teams grow or projects shift, folders and permissions can be reorganized on the fly – no need to start from scratch.
Pro Tip: Many organizations use folders to mirror their internal structure, but you can also get creative – organize by campaign type, business objective, or even experiment status.
Buckets: Keeping Experiments in Their Own Lanes
While folders help you organize and control access, Buckets (sometimes called “traffic repartition”) are all about managing how user traffic is allocated across experiments. Think of buckets as traffic lanes on a highway – each experiment gets its own lane, so there’s no risk of collisions.
How Buckets Work
Traffic Allocation: By default, you can create up to 10 buckets, each representing 10% of your total user traffic. Assign experiments to specific buckets to ensure they don’t overlap.
Mutual Exclusivity: Experiments in different buckets never see the same users, so results are clean and reliable.
Planned Flexibility: While the default is 10 buckets, future updates will allow you to customize the number of buckets and the percentage of traffic allocated to each.
Why Buckets Matter
No Overlap: Run multiple experiments at the same time – on the same page or feature – without worrying about interference.
Reliable Results: By keeping experiments mutually exclusive, you avoid skewed data and can trust your insights.
Enterprise-Ready: Especially valuable for organizations with multiple teams running simultaneous experiments.
Why This Matters for Your Team
Folders & Buckets aren’t just “nice-to-have” features – they’re essential for any organization looking to scale experimentation without losing control. They help you:
Stay organized as your program grows.
Keep sensitive experiments secure and compliant.
Empower teams to work independently without stepping on each other’s toes.
Deliver reliable, actionable insights by preventing experiment overlap.
As digital experimentation becomes a core business function, tools like Folders & Buckets are what separate the leaders from the laggards.
Ready to Get Organized?
If you’re struggling with a cluttered experimentation environment or worried about experiment overlap, it’s time to explore what Folders & Buckets can do for you. Customer Success Manager for more information, and see how easy it is to bring order – and results – to your experimentation program.
Experiment boldly. Organize smartly. Grow faster.
Want to learn more? Check out our documentation (folders/buckets) or contact us for a personalized demo.
How can I stay organized when running lots of experiments?
AB Tasty is built for big teams running many experiments. We offer our users a clear folder structure to group experiments by team, project, product line, or page type, and apply granular permissions so people only see the campaigns relevant to them. This reduces clutter, limits mistakes, and keeps your experimentation environment manageable as you scale.
What are folders in an A/B testing or experimentation platform?
Folders act like a filing cabinet for your tests: you can group experiments in ways that match your organization (e.g., by team, sprint, product, or page type) and assign viewer, editor, or admin roles per folder to control who can see and edit each campaign.
What are buckets in an experimentation platform?
Buckets (or “traffic repartition”) are a way to divide user traffic into separate lanes. Each bucket gets a portion of traffic (e.g., 10%) and experiments assigned to different buckets don’t share users, which keeps tests mutually exclusive.
Which A/B testing solutions help teams stay organized at scale?
The most effective solutions offer both structural tools (like folders with role‑based access) and traffic management features (like buckets for mutual exclusivity), like AB Tasty. Together, these help large organizations keep experiments secure, organized, and analytically sound as their programs grow.
The use of agentic AI for A/B testing is literally a game-changer for marketing teams, making it much easier to scale testing and experimentation programs. But with some companies making bold claims about what their AI can do, it can be hard to know just what to believe. So let’s look at how the competition’s AI really stacks up against ours.
First, let’s look at our agentic AI. Launched in November 2025, Evi is AB Tasty’s AI-powered marketing agent designed for evidence-based decision making. It transforms complex data into clear, actionable strategies for repeatable, measurable results and ensures every step you take is grounded in evidence.
But Evi isn’t just one tool. Evi is a suite of intelligent AI agents integrated throughout the entire AB Tasty platform, all optimized for specific tasks.
Agent
Function
Description
Evi Ideas
Ideation
Scans pages and generates data-backed ideas for new tests based on visual and contextual input. It uses AB Tasty’s proprietary data and UX principles.
Evi Hypothesize
Hypothesis creation
Uses a checklist of essential elements to help you create well-structured hypotheses with clear objectives. Assigns quality scores, highlights gaps and suggests edits.
Evi Content
Visual editor
Turns natural language prompts into precise on-page edits (HTML/CSS/JS) with no coding required.
Evi Analysis
Post-test analysis
Analyzes campaign data, delivers clear, actionable insights. Highlights winning variations and breaks down why they drive transactions.
Evi Feedback
Qualitative analysis
Analyses Net Promoter Score (NPS) and Customer Satisfaction (CSAT) feedback. Quickly identifies key themes and provides actionable insights from customer comments.
Evi Explore
Revenue insights
Powered by our patented metric, RevenueIQ, provides real revenue projections per visitor / per month with confidence intervals. Let’s you see what each test is worth before you launch.
Evi Formula
Catalog attributes (R&M)
Self-serve tool for creating catalog attributes using natural language prompts.
So how does Evi compare to the AI used by our main competitors?
Evi vs. Optimizely Opal
Opal is the name of Optimizely’s suite of AI tools integrated throughout their platform. It’s not a standalone product, but rather a collection of different AI agents threaded across their entire product suite, which, along with Experimentation, also includes CMS, CDP, and Commerce.
Indeed, most of Opal’s AI agents are actually focused on CMS, CDP, and Commerce rather than Experimentation. One potential drawback for customers is that many of these AI features are tied to using the entire Optimizely tech stack. Rather than talk about Opal’s features for other areas, let’s look at what AI features Opal does have specifically for Experimentation:
Feature
Description
Test ideation
Generates ideas for new experiments based on URLs and brand tone.
Variation editor
AI-assisted creation of test variations based on Google Gemini.
Campaign creation
Creates containers for both web and feature experimentation.
Variable suggestions
Suggests flag variables and variations in feature experimentation.
Chat-based data exploration
Allows conversational exploration of test data.
Results summarization
Summarizes test results and provides directional guidance.
Experiment advisor agents
These include a personalization advisor, experiment planner, and results summarizer.
Experiment scorecard
Scores experiments from the analytics interface.
Opal’s AI agents that are used specifically for testing and experimentation are very comparable to those of Evi. Both have dedicated AI agents for ideation, editing, and analysis. However, Evi also includes our proprietary RevenueIQ analysis and can leverage AB Tasty’s other AI features, EmotionsAI and Wandz for targeting and segmentation.
Some of Opal’s features also appear to be standard statistics features that have been rebranded as AI (e.g. multi-armed bandits and sequential testing).
Key differences between AB Tasty’s Evi and Optimizely Opal
Price: Evi’s AI features are included in all contracts at no additional cost. Opal is a paid add-on that costs around US$30,000 extra.
Speed: Evi’s AI editor is based on OpenAI and proven to be faster than that of Opal, based on Google Gemini.
AI Targeting: Evi can leverage our other AI features, EmotionsAI and predictive targeting (Wandz) for AI-based segmentation. Opal has nothing comparable.
Revenue Analysis: Evi Explore is based on our patented RevenueIQ metric for ROI projections. Again, Opal has no equivalent.
Experimentation focus: Evi is 100% focused on testing and experimentation. Most of Opals AI agents are designed for CMS/CDP/Commerce.
Evi vs. Kameleoon PBX
Kameleoon PBX (Prompt-Based Experimentation) is an AI-powered tool that allows users to generate A/B tests directly from natural language prompts. It is positioned as an all-in-one AI agent for test generation, fully integrated with Contentsquare.
Here is a list of PBX’s key AI features:
Feature
Description
Prompt-based test generation
Users write prompts describing what they want to test, PBX then generates the necessary code/variation.
Contentsquare integration
Turns behavioral insights from Contentsquare into A/B tests.
Automatic site analysis
Upon URL integration, 3 to 4 AI agents analyze the site’s structure (HTML/CSS/JS) giving prompts strong contextual awareness.
Figma integration
(Currently a Beta feature) Allows the uploading of mockups from Figma, reducing errors in banner creation and saving time in implementation.
Code generation
Generates HTML, CSS, and JS code for experiments.
Unlike Evi’s suite of specialized AI agents, PBX is a single generalist AI agent. This provides you with a limited amount of control and can make it hard to iterate. Kameleoon claims that by using PBX specifically, its customers can build tests faster, more accurately, and at less cost per test. But the reality is that these improvements aren’t specific to PBX, all vendors with agentic AI see similar positive impacts for their customers.
Key differences between AB Tasty’s Evi and Kameleoon PBX
Price: Evi’s AI features are included in all contracts at no additional cost. Like Opal, PBX is a paid add-on.
Usage: All AB Tasty customers can make unlimited use of Evi, while use of PBX is based around credit quotas.
Architecture: Evi is a suite of different specialized AI agents. PBX is a single generalist agent.
Speed: Evi has a prompt response time of around 30 seconds, compared to up to 3 to 4 minutes for PBX.
Advanced segmentation: Evi can leverage AB Tasty’s other AI features like EmotionsAI for advanced segmentation. However, like Opal, PBX has nothing comparable.
Structured output: Evi supports structured output (JSON, rollback, versioning), while PBX makes no mention of whether this is the case.
Production quality: Evi is fully production-ready, while some customers have reported that PBX has QA issues and webperf impact.
Your search bar is the fastest path to conversion. Yet for too long, site search solutions have been frustrating: complex, rigid, and disconnected from what e-commerce and product teams actually need to do their jobs.
Even customers using the most expensive search solutions told us they were dissatisfied. The tools were too complex, built for developers instead of business teams, and required painful manual work to sync marketing campaigns across their site.
We heard you. Loud and clear.
So we rebuilt Site Search from the ground up. The revamped AB Tasty Site Search is now available—designed to drive revenue, empower your teams, and integrate seamlessly into your workflow.
Semantic Search That Understands Intent
The biggest transformation? We’ve eliminated the endless synonym nightmare.
Traditional search engines rely on keyword matches. If a customer searches for “running shoes” but your product catalog says “athletic footwear,” you lose the sale. Fixing this meant maintaining massive synonym lists that were tedious to build and impossible to keep current.
AB Tasty Search uses a hybrid engine that blends keyword matching with semantic AI. The system understands the meaning behind queries, not just the exact words typed.
The impact:
Zero “No Results” dead ends: Semantic AI delivers relevant results even when users misspell, use natural language, or describe products contextually
No manual synonym maintenance: The system handles variations automatically
Real revenue gains: Early clients report a +11.6% increase in transactions and a +6.9% lift in add-to-cart rates
Understanding of user needs: We leverage AI to understand user queries without requiring a complex setup.
Built for Business Teams, Not Just Developers
We know the pain of being bottlenecked by engineering resources. Need to spotlight a product for a campaign? Waiting days (or weeks) for dev time kills momentum.
The new AB Tasty Search puts control where it belongs: in the hands of merchandisers and marketing teams.
We provide intuitive low-code/no-code tools that enable rapid iteration without developer dependency. This agility is delivered through two key mechanisms:
Widget-based deployment: You can deploy the search functionality quickly using a simple widget that adds an overlay when a user clicks the search bar. This widget-based deployment allows you to challenge your existing solution in days, not months, enabling faster time to value.
Intuitive merchandising controls: Merchandisers can optimize ranking, spotlight key products, and tailor results to user behavior. You can easily boost or bury products based on any catalog attribute, set query-specific redirections, and make real-time product ranking adjustments.
The result? True autonomy and agility for non-technical teams.
The Strategic Advantage: A Unified Platform for Discovery
Here’s where it gets really powerful: the revamped Search isn’t a standalone tool. It’s part of your complete optimization platform.
Search, Recommendations, and Merchandising now share the same rules, the same interface, and the same data. That marketing campaign you’re launching? Apply your merchandising logic once, and it works consistently across product discovery, search results, and recommendation widgets.
Consistency across channels: Same rules across Search, Merchandising, and Recommendations – no more manual syncing.
Personalization: Leverage your unified customer data to tailor search results by user type and behavior.
Simplified implementation, stack, and cost: One vendor, one dashboard, one connection to your catalog, and one source of truth for all your optimization activities.
This unified approach allows you to deploy product discovery as a global strategy, challenge it with unified KPIs and a dashboard, and optimize holistically.
What’s Next? Conversational Discovery
The search bar is evolving, and so is AB Tasty. We are already looking beyond traditional search to deliver a truly multi-modal experience that blends conversational, discovery, and keyword search into one seamless journey.
And with our approach to personalization, every search result can be personalized in real time based on user behavior, preferences, and context—delivering the right products to the right people at exactly the right moment. Read more about AdaptiveCX here.
Our upcoming Shopping Assistant will bring an AI-powered chat interface to your site through the same easy widget deployment. Natural language conversations will guide shoppers to the right products, increasing conversion while reducing returns.
Get Started with AB Tasty Site Search
The revamped AB Tasty Search module is generally available, ready to replace basic native search solutions (like Shopify or Salesforce) or replace your complex, developer-dependent tools.
If you are looking to eliminate “no results” experiences, empower your merchandising teams, and unify your product discovery strategy under one intelligent platform, the time to explore the new AB Tasty Search is now.
Talk to your AB Tasty representative to start testing the new Search module today. The best way to know if it works for you? Test it.
Understanding your customers’ paths? Not easy. Each person arrives with their own reason for visiting your site and takes their own route through your pages.
So how do you gain real insights to improve usability and spot buying trends?
Start with building a customer journey map.
In this blog, we’ll walk you through what a customer journey map is, how to build a customer journey map, which templates work best for your customer journey map, and how to put them into action. Let’s get started!
What is a customer journey map?
A customer journey map is a visual tool that shows how customers interact with your business or website—from start to finish.
It helps you spot where things aren’t working and improve the overall experience.
Think of it as a story told visually. It maps out:
What customers do
What they think
How they feel
At the heart of the map are touchpoints—specific moments where customers interact with your brand. Maybe they’re researching a product, making a purchase, waiting for delivery, or requesting a return.
Each touchpoint can be positive, neutral, or negative from the customer’s perspective. Your job? Make more of them positive.
Customer journey mapping requires a mix of hard data, customer feedback, and creative thinking. No two maps are the same—and that’s the point. Every business is different.
7 Reasons Why Use Customer Journey Maps
Customer journey mapping isn’t just a nice-to-have—it’s a strategic tool that drives real business results.
Here’s why it matters:
1. See Through Your Customers’ Eyes
Journey maps help you step into your customers’ shoes. You’ll understand their motivations, expectations, and frustrations at every stage—not just what they do, but why they do it.
That empathy translates into better decisions, smarter strategies, and experiences that actually resonate.
2. Spot and Fix Pain Points Fast
Every journey has friction. Your checkout process might be too complicated, your search function delivers the wrong results, or customers can’t find help when they need it.
Customer journey mapping reveals these bottlenecks so you can address them before they cost you customers.
3. Build Loyalty That Lasts
When customers feel understood and valued, they stick around. By removing barriers and meeting needs at every touchpoint, you strengthen the emotional connection between your brand and your audience. That connection drives repeat purchases and long-term loyalty.
In fact, a 5% increase in customer retention can lead to a 25% increase in profits.
4. Personalize at Scale
Not all customers are the same—and your experience shouldn’t treat them that way. Journey maps highlight individual preferences and behaviors, enabling you to tailor messaging, product recommendations, and support to each person.
Personalization increases purchase likelihood and makes customers feel like you actually get them.
5. Align Your Entire Team
Customer journey mapping breaks down silos. When marketing, product, sales, and support all work from the same map, everyone understands the customer’s perspective and how their work impacts the overall experience.
That shared understanding leads to better collaboration, faster problem-solving, and a more cohesive brand experience.
6. Make Smarter, Data-Driven Decisions
Journey maps aren’t just pretty visuals—they’re strategic tools backed by real data.
They guide decisions about where to invest, what to test, and which initiatives will have the biggest impact on customer satisfaction and business growth.
7. Drive Innovation and Stay Ahead
Customer needs evolve. Markets shift. New competitors emerge.
Regularly reviewing and updating your customer journey map helps you spot emerging trends, changing preferences, and new opportunities before your competitors do. It keeps your brand agile, innovative, and ready to adapt.
The Heart of Customer Journey Mapping: Buyer Personas
Buyer personas are fictional characters based on real customer data. They represent your audience in a way that’s relatable and actionable.
Most projects create between three and seven personas—and each one gets its own customer journey map. Why? Because different customers have different needs, goals, and pain points. A persona helps you walk in their shoes and design experiences that truly resonate.
Personas aren’t perfect replicas of real people. They’re broad representations that guide smarter decisions.
Who Benefits from a Customer Journey Map?
Short answer: everyone.
Customer satisfaction drives loyalty more than ever. People are more informed, more demanding, and more willing to shop around.
A well-designed customer journey map helps you:
Highlight problems customers face
Build stronger relationships with your brand
Keep customers at the center of every decision
Once your map is ready, your entire team—from marketing to product to support—can use it to stay aligned and customer-focused.
Bringing Your Whole Business Together
Customer journey mapping isn’t just for your customer-facing teams. It brings everyone together.
When you map out touchpoints, departments that don’t usually interact with customers start to see how their work affects the experience. That’s powerful.
For example:
How easy is it for someone to find return instructions on your site?
How fast do they get a response when they need help?
What happens after the purchase?
Traditional marketing often stops at checkout. But the customer journey doesn’t. Post-purchase experience matters just as much—and your map should reflect that.
How to Map the Customer Journey Visually?
A customer journey map gives you a clear picture of your customers’ experiences from their point of view.
To create one, focus on two things:
Defining customer goals – What are they trying to accomplish?
Understanding their nonlinear journey – Customers don’t move in straight lines
By mapping every interaction, you’re identifying opportunities to delight your customers and craft smarter engagement strategies.
According to Aberdeen Group, 89% of companies with multi-channel engagement strategies retained their customers—compared to just 33% of those without one.
You can build your map using:
Excel sheets
Infographics
Diagrams
Illustrations
Customer journey maps also help with:
Retargeting with an inbound mindset
Reaching new customer segments
Building a customer-first culture
All of this leads to better experiences, more conversions, and stronger revenue.
There are four main types of customer journey maps. Each highlights different behaviors and serves different goals.
1. Current State Template
Shows what customers currently do, think, and feel. Great for spotting pain points and making incremental improvements.
2. Future State Template
Focuses on what customers will do, think, and feel. Useful for planning new products, services, or experiences.
3. Day in the Life Template
Similar to the current state map, but broader. It looks at how customers behave with your brand and your competitors. Perfect for uncovering unmet needs.
4. Service Blueprint Template
Starts with a simplified current or future state map, then adds the internal processes, people, and tech behind the experience. Helps you see the full picture—front and back.
How to Create a Customer Journey Map in 7 Steps ?
Creating customer experience journey maps takes time, but the payoff is worth it. Here’s how to do it.
Step 1: Create Buyer Personas
Start with a clear objective. Who is this map for? What are you trying to learn?
Building personas is the most time-consuming part—but also the most important. You’ll need:
AB Tasty is a best-in-classexperimentation platform that helps you test variations, personalize experiences, and convert more customers—fast. With AI and automation built in, you can optimize the digital experience with confidence.
Once your map is live, review and update it regularly. Customer journeys evolve—and so should your map.
How to Collect Journey Mapping Data?
Great customer experience journey maps are built on solid data—not assumptions. You’ll need a mix of qualitative insights (the “why” behind behavior) and quantitative metrics (the “what” you can measure).
Here’s how to gather both:
1. Qualitative Data: Understanding the “Why”
Qualitative research helps you uncover motivations, emotions, and pain points that numbers alone can’t reveal.
Customer Interviews
Have real conversations with your customers. Ask about their experiences, what frustrates them, and what they love. These in-depth discussions provide rich, nuanced insights.
Surveys
Use open-ended questions to gather feedback on specific parts of the journey. Keep them short and focused to get honest, actionable responses.
User Testing
Watch how people interact with your website or product in real time. Tools like usability tests reveal where users get stuck, confused, or frustrated.
Mystery Shopping
Experience your own customer journey firsthand. Walk through every step—from discovery to purchase to support—and see what works and what doesn’t.
Support Transcripts
Review customer service conversations to identify recurring issues and common questions. These transcripts are goldmines for understanding pain points.
2. Quantitative Data: Tracking the “What”
Quantitative data gives you measurable, trackable insights that help you validate assumptions and monitor progress over time.
Website Analytics
Tools like Google Analytics show you how customers navigate your site, where they drop off, and which pages drive the most engagement.
See exactly how users interact with your pages—where they click, how far they scroll, and where they hesitate. Tools like Hotjar and Contentsquare make this easy.
Conversion Funnels
Track how customers move through key stages of the journey and identify where they abandon the process.
Customer Satisfaction Scores
Metrics like Net Promoter Score (NPS) and Customer Satisfaction (CSAT) measure loyalty and satisfaction at different touchpoints.
CRM Data
Your CRM system (like Salesforce or HubSpot) holds valuable information about customer interactions, purchase history, and behavior patterns.
Social Media Listening
Monitor what customers say about your brand on social platforms. This reveals sentiment, trends, and unfiltered feedback.
Email Campaign Metrics
Analyze open rates, click-through rates, and conversion rates to understand how customers engage with your messaging.
Support Ticket Volume
Track common issues and complaints to identify systemic problems in the customer journey.
Best Practices for Journey Map Data Collection
Combine Both Types of Data
Qualitative insights explain why customers behave a certain way. Quantitative data shows you what they’re doing. Together, they give you the full picture.
Test Your Assumptions
Don’t rely on guesses. Validate your hypotheses about customer behavior through research and real data.
Involve Stakeholders
Gather input from marketing, sales, product, customer service, and leadership. Each team has unique insights that make your map more accurate and actionable.
Keep It Current
Customer behavior changes. Markets evolve. Your journey map should too. Update it regularly to stay relevant and effective.
Customer Journey Map Examples
Customer journey maps come in all shapes and sizes. Some look like works of art. Others are simple sketches. What matters is clarity.
Here are some real-world examples of customer journey mapping in action:
1. David Jones: Simplifying Account Access
David Jones, a major Australian retailer, mapped their customer journey to understand how shoppers interacted with their account features during the buying process.
Through testing and personalization, they made it easier for customers to access their accounts, track orders, and manage preferences.
