Digital experimentation has matured. Where A/B testing was once handled by a handful of specialists, today it’s a team sport — involving marketing, product, UX, engineering, and data. Organizations now need platforms that connect these roles, reduce friction, and enable collective decision-making.
This is where AB Tasty stands apart. More than an experimentation tool, it is a collaborative ecosystem, designed to help companies run impactful tests at scale while empowering every contributor in the process. From idea generation to final reporting, AB Tasty removes silos and strengthens alignment — a critical ingredient for a successful CRO program.
A Platform Built for Cross-Functional Workflows
Modern experimentation involves teams with different expertise and expectations. AB Tasty addresses this diversity through a unified platform that brings everyone together.
1. Clear Governance and Team-Based Visibility
Large organizations often struggle with visibility: too many tests, too many markets, too much noise. AB Tasty’s advanced RBAC system solves this by assigning precise roles and allowing teams to create custom folders and views. A French editor only sees French campaigns; a central CRO manager sees everything; a developer accesses only what they need.
This structure reduces operational clutter and protects the integrity of local workflows, while still enabling global oversight.
2. Collaboration at Every Step of the Experiment Lifecycle
Where AB Tasty excels is its ability to facilitate teamwork throughout the entire testing process.
Before a Campaign
Teams use the Ideas Backlog to surface opportunities and prioritize them together, while the Learning Library accelerates strategy by making past learnings accessible across markets.
During a Campaign
The no-code visual editor empowers marketers, while developers leverage VS Code. Comments can be added anywhere — in the editor, creation flow, or reports — with tagged users notified instantly. Preview and QA links make cross-team collaboration effortless.
After a Campaign
AB Tasty’s segmentation features and its powerful Data Explorer allow analysts to go deep, while marketing teams can still interpret results intuitively. Reports can be shared externally via secure links or exported automatically to Notion, BI systems, or Slack channels. Visibility becomes effortless and organization-wide.
3. Deep Integrations Strengthen Collective Intelligence
Collaboration is not limited to the experimentation platform itself. AB Tasty integrates with tools teams already use daily:
Slack: receive notifications when campaigns go live or when new learnings are added
Notion: synchronize campaign KPIs and reports automatically into team workspaces
BigQuery, Looker, Metabase: power custom dashboards
GA4, Contentsquare, FullStory: enrich analysis with behavioral and analytics data
And with Microsoft Teams coming soon, AB Tasty is extending its collaborative reach even further.
4. A True “One Platform” for Experimentation, Personalization, and Feature Rollouts
Cross-team alignment is reinforced by AB Tasty’s unique combination of client-side experimentation and Feature Experimentation & Rollout (FE&R). Product teams and engineers can gradually deploy new features, run server-side tests, and secure releases through progressive rollout and rollback automation. Meanwhile, marketing and CRO teams continue to run client-side tests on the same unified platform.
Everyone operates within the same environment, driving shared KPIs and shared business outcomes. And this collaborative foundation is amplified by Evi — AB Tasty’s evidence-based AI agent.
5. How Evi Enhances Collaboration Throughout the Experiment Lifecycle
Evi acts as a shared intelligence layer that supports every role involved in experimentation — ensuring alignment, speed, and evidence-based decisions at every step.
Before launching a test
Evi Ideas generates new experiment opportunities
Teams align faster on hypotheses grounded in evidence
Evi Content creates consistent messaging across markets
During the campaign
Evi provides contextual guidance directly in the workflow
Teams can iterate faster and reduce dependency loops
After the campaign
Evi Analysis turns raw results into clear, actionable insights
Everyone sees the same interpretation of data
Learnings become easier to share and apply across markets
Result: A more autonomous, aligned, and collaborative experimentation program — powered by shared intelligence rather than siloed expertise.
AB Tasty continues to strengthen its collaborative features, with upcoming developments. Stay tuned! These improvements move AB Tasty closer to its long-term vision: a platform that not only enables experimentation but also unlocks organizational intelligence.
Conclusion: Collaboration Is the New CRO Advantage
Companies win with experimentation when they democratize it — when insights circulate openly, when accountability is shared, and when tools empower collaboration instead of slowing it down.
AB Tasty doesn’t just enable experimentation; it turns it into an organizational capability. A place where teams align faster, learn continuously, and make decisions grounded in evidence rather than intuition.
In a world where speed and cross-functional execution define competitive advantage, AB Tasty provides the collaborative foundation businesses need to accelerate growth.
Danielle Harvey shares how travel customers are using different channels, why testing doesn’t always have to end in success, and how travel companies can integrate AI to provide a more engaging customer experience.
Currently Vice President, Industries, Partnerships & Emerging Products at Quantum Metric, Danielle Harvey, has a long experience in the travel industry. She also spent 11 years at one of the world’s largest hotel brands, Wyndham Hotels & Resorts, driving a data-driven approach to optimizing customer experience. With roles including digital acquisition, voice of the customer, CRM, experimentation, and digital analytics, she has a unique understanding of the travel customer journey.
Danielle Harvey spoke with AB Tasty’s Head of Marketing and host of The 1000 Experiments podcast, John Hughes, about the importance of connecting channels in the travel industry, using testing to understand the customer journey, and how brands can best harness the power of AI.
Here are some of the key takeaways from their conversation.
Let customers do what they want where they want
COVID forced the travel industry to undergo an accelerated digital transformation. And travel customers now want a seamless cross-channel experience when booking. This is especially true when people frequently use different platforms at different stages of their buying journey and make multiple visits to your website before making a booking.
“We did some benchmark data and 75-80% of traffic in travel is on mobile at this point, but only about 25% of bookings are. It’s a heavy research channel, a day-of-travel channel, but not necessarily where people are comfortable purchasing yet,” says Danielle.
Enabling customers to transact in their channel of preference and connecting different channels therefore becomes vital. This provides immediate benefits for the customer but also operational efficiencies for providers.
Omnichannel may have been little more than a buzzword a few years ago. But with the true adoption of digital technology and improved methods of data collection, connecting those experiences is becoming more of a reality.
“A lot of travel can still be pretty siloed, but your customers don’t care,” explains Danielle. “They expect that your teams are speaking to each other, that there’s an overarching strategy.”
10 essential ideas to help travel brands win bigger.
Even flat and failed tests can be learning experiences
Testing and experimentation don’t always have to be successful to provide you with valuable information to help you improve the customer experience. An example that Danielle gave was testing customer ratings and reviews on a website.
“Some of the most interesting testing I did was around reviews. Because the assumption was that if you get those out there on the site, they should really have an impact,” says Danielle.
But when almost a quarter of the people researching travel will typically visit your website as least five times before booking, it’s likely that they’re getting much of their information from other sources.
“It was always interesting that whenever we tried testing reviews, they didn’t really move the needle. So, your website is often not the only place people are going to go for information,” notes Danielle.
But this flat result helped drive the realization that while adding reviews might not have a direct financial impact, they were important for transparency. And at the same time, they made things easier for the customer.
And just because a test has failed, that doesn’t mean it shouldn’t help to inform your strategy going forward. The key is to try and understand what happened and learn from that.
“Over time, I would typically see a 50/50 win to fail rate. But my focus on failed tests was always what do we learn from this, digging into the reason why it failed and then building a pipeline of testing and experimentation off of that,” says Danielle.
Use AI to improve the customer experience
AI-powered tools can create time efficiencies for travel providers and provide valuable context about customer intent. And many travel brands are using AI to help their employees service the customer faster.
“We’re doing some cool stuff at Quantum where an AI chat component will send a summary to a support agent who can immediately see what the customer was trying to do, rather than putting the burden on the customer to repeat themselves,” explains Danielle.
Integrating AI can also be extremely valuable for people involved in testing and experimentation.
“A lot of the excitement around AI, especially in things like personalization, is that you don’t need to come up with ideas yourself and test them, but ideally some of that is automated for you,” says Danielle.
If you launch a test and don’t specifically track certain behaviors, for example, it’s often hard to know how a user might interact with it. By using AI to auto capture data, you can watch what users did and use heat maps to see where they were engaging.
There’s also an increasing focus from both customers and travel providers on self-service. But many brands are still hesitant to have a lot of AI facing the customer. The key is finding the right balance.
“The unique thing with travel and hospitality is there is always a human element. You don’t want to digitize it completely,” advises Danielle. “You’re ideally delivering a nice experience as well.”
What else can you learn from our conversation with Danielle Harvey?
The long-haul effect: How the travel customer journey differs from that of e-commerce.
Voice of the customer: The importance of turning qualitative feedback into quantitative data.
On brand: Some of the challenges involved in testing across different brand websites
Experience over things: Why travel will continue to be a priority for many people going forward even though it might look different.
About Danielle Harvey
Danielle Harvey is Vice President, Industries, Partnerships & Emerging Products at Quantum Metric. Passionate about the travel industry, she spent 11 years prior to this leading digital and analytic teams at Wyndham Hotels & Resorts and has also worked for the Avis Budget Group.
About 1,000 Experiments Club
The 1,000 Experiments Club is an AB Tasty-produced podcast hosted by John Hughes, Head of Marketing at AB Tasty. Join John as he sits down with the experts in the world of experimentation to uncover their insights into what it takes to build and run successful experimentation programs.
The Problem: The High Cost of Experimentation Amnesia
In digital optimization, we often obsess over velocity—how fast can we test? But this focus masks a deeper, more expensive problem: Experimentation Amnesia.
At AB Tasty, an analysis of over 1.5 million campaigns revealed a startling trend. While thousands of tests are launched daily, the specific context—why a test won, what surprised us, and the strategic lesson learned—often evaporates the moment the campaign ends.
It vanishes into a 250-slide PowerPoint deck that no one opens again. It disappears into a Slack thread. Or, most painfully, it walks out the door when your CRO Manager or agency partner moves on to their next opportunity.
If you are running tests but not archiving the insights in a retrievable way, you aren’t building a program; you’re just running in circles. It’s time to shift your focus from Execution to Knowledge Management.
The Hidden Cost of “One-and-Done” Testing
The digital industry is notorious for its high turnover. On average, internal digital teams change every 18 months and agencies rotate every two years.
In traditional workflows, knowledge is tied to people, not platforms. When a key manager leaves, they take their “mental hard drive” with them.
This is the “Knowledge Drain.” It is the silent budget killer of CRO programs.
Every time you repeat a test because you couldn’t find the previous results, you are paying double for the same insight. Every time you lose the context of a winning test (i.e., you know that it won, but not why), you lose the ability to iterate and double your gains.
This is why the most mature experimentation teams are moving away from simple testing tools and adopting Program Management platforms that secure their knowledge.
The Solution? AB Tasty’s new Learnings Library.
We designed this feature to serve as a centralized, searchable repository that lives directly where your experiments do. It acts as the institutional memory of your digital team, ensuring that every test—whether a massive win or a “flat” result—contributes to a permanent asset library.
Context is King: Why AI Can’t Replace the Human “Why”
In an era where everyone is rushing to automate everything with AI, you might ask: “Why can’t an AI just write my test conclusions?”
While AI is powerful for analyzing raw numbers, it lacks business context. An AI can tell you that “Variation B increased transactions by 12%.” But it cannot tell you why that matters to your strategy.
Was that 12% expected?
Was it a shocking surprise that disproved a long-held internal belief?
Did it cannibalize another product line?
AB Tasty’s Learnings Library is designed to capture Qualitative Intelligence. It prompts your team to manually qualify results with human tags like “Surprising” or “Expected.” It asks for the narrative behind the numbers.
This human layer is critical. A “failed” test (one that produced no uplift) is often more valuable than a win, provided you document the lesson. By recording, “We learned that our users do not care about social proof on the cart page,” you create a defensive asset. You prevent future teams from wasting budget on that specific hypothesis again.
Visual History: The Power of “Before and After”
One of the biggest friction points in reporting is visual documentation. How much time does your team spend taking screenshots, cropping them, pasting them into PowerPoint, and trying to align the “Control” vs. “Variation” images?
Our Learnings Library automates this friction. It should allow you to upload your screenshots and automatically generate a Comparison View—a visual “Before and After” slide that lives alongside the data.
This visual history is vital for continuity. Two years from now, a spreadsheet number won’t spark inspiration. But seeing the exact design that drove a 20% increase in conversions? That is instant clarity for a new Designer, Developer, or Strategist.
Conclusion: Stop Renting Your Insights
If your testing history lives in the heads of your employees or on a local hard drive, you are effectively “renting” your insights. The moment that employee leaves, the lease is up, and you are back to square one.
It is time to own your knowledge.
Don’t let your next great insight slip through the cracks. Start building your library today.
FAQs: Learnings Library
What is AB Tasty’s Learnings Library?
Our Learnings Library is a centralized digital repository that archives the results, visual history, and strategic insights of every A/B test run by an organization. Unlike static spreadsheets, it connects data (uplift/downlift) with qualitative context (hypotheses and observations), transforming individual test results into a permanent, searchable company asset
How does staff turnover impact A/B testing ROI?
Staff turnover creates a “Knowledge Drain.” When optimization managers leave without a centralized system of record, they take valuable historical context with them. This forces new hires to “restart” the learning curve, often leading to redundant testing (paying for the same insight twice) and a slower velocity of innovation.
Should I document “failed” or inconclusive A/B tests?
Yes. A “failed” test is only a failure if the lesson is lost. Documenting inconclusive or negative results creates “defensive knowledge,” which prevents future teams from wasting budget on the same disproven hypotheses. A robust Learning Library treats every result as a data point that refines the customer understanding.
How do I stop my team from re-running the same A/B tests?
The most effective way to prevent redundant testing is to implement a searchable timeline of experiments that includes visual evidence (screenshots of the original vs. variation). This allows any team member to instantly verify if an idea has been tested previously, under what conditions, and what the specific outcome was.
What is the best platform for scaling a CRO program?
Scaling a program isn’t just about running more tests; it’s about running smarter tests. Unlike competitors that focus on “gadget” features (like AI text generation), AB Tasty invests in Program Management infrastructure. By combining execution with a native Knowledge Management system, AB Tasty allows your program to compound its value over time, rather than resetting every year.
Let’s be honest… Most KPI dashboards are where good intentions go to die. They’re meticulously built, packed with charts, and then they’re mostly ignored. Teams are drowning in data but starved for real insight. Why? Because the framework is broken. It’s either tracking metrics that don’t matter, it’s too slow to be useful, or it’s so complex that nobody knows what the numbers are actually telling them.
This creates a frustrating cycle. Marketing, product, and engineering teams work hard on new ideas, but they can’t prove their impact. The result is a culture of guesswork, not growth. But it doesn’t have to be this way. At AB Tasty, we see optimization as a journey of constant learning, where every experiment moves you forward. A great KPI framework is your guide on that journey. It’s not a static report; it’s a dynamic tool that turns raw data into a clear story of progress, aligns your teams around shared goals, and gives you the confidence to make the next move. It’s time to build a framework that your teams will not only use but will champion.
Step 1: Identify KPIs that are actually key
The biggest trap in measurement is vanity. We track clicks, pageviews, and time-on-page because they’re easy to see and feel like progress. But do they connect to the bottom line? Often, they don’t. A truly effective framework starts by asking the big questions first. What are the core outcomes that drive our business forward?
Forget the firehose of data for a moment and focus on your ultimate destination. Are you trying to:
Increase customer lifetime value?
Improve user retention and reduce churn?
Boost average order value?
Drive qualified leads for the sales team?
Once you have your high-level business objective, you can work backward to identify the Key Performance Indicators (KPIs) that directly influence it. These are your true north stars. If your goal is to grow revenue from your e-commerce platform, your key indicators aren’t just traffic numbers. They’re metrics like purchase rate, average order value (AOV), and overall revenue.
This is why we’ve built our platform with these meaningful goals in mind. Right out of the box, you can set up goals in AB Tasty that cut straight to business impact. These include:
Transaction goals, such as AOV, purchase rate, and revenue.
Action tracking, which measures critical user interactions like clicks, scroll depth, and element visibility.
Browsing behavior, to understand revisits, bounces, and pageviews in a meaningful context.
By starting with KPIs that are directly tied to business outcomes, you create a shared language. Your product team’s experiments with the checkout flow, marketing’s new campaign, and the CRO team’s homepage tests all point toward the same measurable goal. You’re no longer just running tests; you’re driving tangible progress.
Step 2: Measure what makes you unique with custom metrics
Standard metrics are a great starting point, but your business isn’t standard. Your user journey has unique steps, critical interactions, and “aha!” moments that generic KPIs will never capture. Maybe it’s a customer using your product configurator, engaging with a new video series, or filling out a multi-step form. These are the interactions that define your unique customer experience, and you need to measure them.
This is where the grit comes in. It takes determination to go beyond the easy-to-track metrics and measure what truly matters. Custom metrics allow you to translate your unique business logic into trackable data points. They answer specific questions like:
“Did users who interacted with our new sizing guide convert at a higher rate?”
“How many people clicked on the ‘request a demo’ button after watching our new feature video?”
“What percentage of users completed all three steps of our new onboarding flow?”
You shouldn’t have to change your user experience to fit your analytics tool. Your tool should adapt to you. That’s why in AB Tasty, you can create Custom Trackings that are directly linked to the DataLayer on your site. This lets you build metrics based on your specific data. You can also create custom trackers with JavaScript, giving your team the flexibility to measure virtually any interaction. It’s even possible to use these custom tracking events to replicate goals from other platforms, like GA4, ensuring consistency across your entire analytics stack.
When you measure what makes you unique, you get insights that your competitors can’t see. You start understanding the why behind the what, turning your data from a simple report into a competitive advantage.
Step 3: Leverage real-time reporting to act fast
The pace of digital is relentless. An insight that’s a week old is already history. For optimization to be effective, the feedback loop between action and insight needs to be as short as possible. If your team has to wait days or weeks for a report on their latest experiment, they’ve already lost momentum. The opportunity to pivot, iterate, or double down on a winner has passed.
A culture of improvement requires a flow of constant learning. This is where real-time reporting becomes essential. When you can see the impact of your changes as they happen, you empower your teams to be proactive, not reactive. They can spot a problem with a new release within minutes, not days. They can see a winning variation pull ahead and make a confident decision to roll it out to all users, capturing its value immediately.
We know that speed is critical. That’s why AB Tasty offers real-time reporting that automatically activates when you push a campaign live. During the initial, most critical phase of a test (up to 1,000 unique visitors or the first 12 hours), your data for every goal and variation is updated every five minutes. We also provide a Live Hits monitoring tool so you can track event data at any moment. This closes the gap between an idea and its outcome, allowing your teams to make smarter, faster decisions fueled by up-to-the-minute data.
Step 4: Understand confidence to make confident decisions
You’ve run a test, and variation B is outperforming the original by 5%. Is it time to celebrate and deploy it to everyone? This is where many teams get stuck. They see a positive lift but are paralyzed by uncertainty. What if it’s just random chance? How “sure” do we have to be?
This is the final, crucial piece of the framework: understanding the story your statistics are telling you. You don’t need to be a data scientist to make a good call, but you do need tools that present statistical confidence in a clear and actionable way.
At AB Tasty, we use Bayesian statistics, which provides direct and intuitive probability statements. Instead of just a confusing p-value, we give you two key things:
The chance to win: This is a straightforward probability that tells you how likely it is that a variation is better than the original. A 98% chance to win means there’s a 98% probability that the variation is the true winner.
A 95% confidence interval: Think of this as a “range of likely outcomes.” If the confidence interval for the gain is [+2%, +8%], we are 95% confident that the true, long-term uplift from this variation is somewhere between 2% and 8%. The remaining 5% represents the margin of error.
This approach removes the ambiguity. It equips your team with the conviction they need to make decisions. It’s not about being 100% certain; it’s about understanding the probability and the potential impact, allowing you to make a calculated business decision. It’s the insightfully sharp wisdom that turns a good idea into proven progress.
Conclusion: Find your better
A KPI framework is so much more than a dashboard. It’s a reflection of your strategy, a tool for alignment, and the engine of your experimentation culture. When you focus on what matters, measure your unique journey, act on insights in real-time, and make decisions with confidence, you create a powerful cycle of continuous improvement.
You stop guessing and start knowing. Your teams feel empowered because they can see the direct impact of their brave ideas. You build a culture that doesn’t just celebrate the wins but learns from every single test. You start your journey to “Find your better.” Your data is already telling this story. It’s time to build the framework that lets you read it.
Frequently Asked Questions (FAQ)
What’s the difference between a KPI and a metric?
Think of it this way: a metric measures a process, but a KPI measures performance against a key business objective. For example, “website traffic” is a metric. But “conversion rate from organic traffic,” when your goal is to increase online sales, is a KPI. All KPIs are metrics, but not all metrics are KPIs.
How many KPIs should we track?
Less is more. A framework with 25 KPIs is a list, not a focus. Aim to identify 3-5 primary KPIs for each major business objective. This forces you to prioritize what truly drives the business forward and keeps your teams from getting lost in the noise.
How do we get our teams to actually use the framework?
Adoption comes from ownership and accessibility. Involve marketing, product, and data teams in the creation process so the KPIs reflect their reality. Then, make the dashboard highly visible and easy to understand. Most importantly, celebrate the insights the framework generates, not just the successful tests. Frame it as a tool for learning, and your teams will embrace it.
What kinds of KPIs are available out of the box in AB Tasty?
You can set up goals at both an account and campaign level. The primary goal types include:
Action Tracking: Clicks, dwell time, element visibility, scroll rate.
Page Tracking: Visits to a specific page or group of pages.
Browsing Behavior: Revisit rate, bounce rate, pageviews per session.
Transaction: Average order value (AOV), purchase rate, total revenue.
DataLayer Goals: Tracking based on variables in your site’s data layer.
Can we create custom metrics?
Yes. You can create Custom Trackings linked directly to your DataLayer, allowing you to build metrics from your unique business data. You can also create custom trackers using JavaScript, which can be used to track specific interactions or even replicate goals from other platforms like GA4.
Is reporting in real-time?
Yes. Real-time reporting activates automatically when a campaign goes live. During the initial phase (the first 1,000 unique visitors or 12 hours), data for each goal and variation are updated every five minutes. We also provide a Live Hits monitoring tool to track event data at any time.
How does AB Tasty report on statistical confidence?
We provide two key figures to make decisions easier. The first is a 95% confidence interval, which gives you a likely range for the true value of the gain. The second is the Chance to Win, which is a direct probability that one variation is better than the other. We use a Bayesian statistical model because it provides these intuitive, actionable results that are easier for teams to understand and act on.
What’s the difference between client-side and server-side testing?
Client-side testing happens in the user’s browser and is ideal for marketing and CRO teams to quickly test visual changes, copy, and layouts without needing developer support. Server-side testing happens on the web server, which is better for product and engineering teams to test more complex functionality, new features, and omnichannel experiences. AB Tasty’s unified platform supports both, allowing teams to work from a single place.
How do we measure the impact of personalization on KPIs?
The key is to always test your personalization efforts. Run an A/B test where Group A sees the standard experience and Group B sees the personalized experience. By comparing the KPIs for both groups—such as conversion rate, AOV, or engagement—you can directly attribute any uplift to your personalization strategy and prove its ROI.
No Compromise with Security: AB Tasty’s Commitment to Safe Experimentation
At AB Tasty, we believe security should never be an afterthought. That’s why we’ve taken a major step forward by removing the use of JavaScript’s eval() function from our platform.
While eval() was once a common way to execute dynamic code for A/B testing, it’s now widely recognized as a security risk—vulnerable to code injection attacks and often blocked by strict Content Security Policies (CSPs).
What does this mean for you?
Our platform is now fully compatible with even the strictest CSPs.
The risk of code injection and related vulnerabilities is dramatically reduced.
You get a safer, more robust experimentation environment—no exceptions or workarounds needed.
What Is eval() and Why Is It Considered Unsafe?
eval() is a native JavaScript function that takes a string of code and executes it as if it were written directly in the script. This flexibility makes it convenient for scenarios where dynamic execution is needed—such as A/B testing, where variations are generated on the fly. However, eval() is often considered a security risk because:
It executes code without validation, making it a potential gateway for malicious scripts.
It is vulnerable to code injection attacks, which can compromise a website’s security.
Many modern security policies, including Content Security Policies (CSPs), explicitly prohibit or restrict the use of eval().
Security at Every Level
Our commitment to security goes far beyond code execution. Here’s how we keep your data and your business safe:
Data Protection: All data is encrypted in transit and stored securely on Google Cloud infrastructure.
Access Management: Only a select, authorized team can access customer data, with all access logged and regularly reviewed.
Authentication & Permissions: We support strong password policies, multi-factor authentication, and role-based access control.
Security-conscious businesses now have an additional reason to choose AB Tasty over other Customer Experience Optimization providers. While some major providers still rely on eval(), our eval()-free approach offers a safer alternative without compromising performance.
By proactively adapting to modern security standards, AB Tasty ensures that our clients can run experiments without worrying about security vulnerabilities or policy restrictions.
The Future of Secure A/B Testing
Security and performance go hand in hand. At AB Tasty, we continuously evolve to meet the highest standards of safety and efficiency. Removing eval() is just one of the many steps we take to provide a secure, high-performance CRO experience.
If you’re looking for a compliant, secure, and high-performing experimentation platform, AB Tasty is the solution. Contact us today to learn more about how we can help you optimize your website—safely and effectively.
FAQs about security and privacy at AB Tasty:
Why is the removal of eval() important for security?
eval() can execute any code, making it a target for code injection attacks. By removing it, AB Tasty eliminates a major security risk and ensures compatibility with strict Content Security Policies.
Will this change affect the performance or flexibility of my experiments?
No. The processing is fully optimized on AB Tasty’s side and completely transparent for you. Tests remain fast, lightweight, and without any impact on page performance.
Do we need to do anything on our IT side?
Nothing at all. No CSP rule to adjust. Integration is now simpler than ever.
What makes AB Tasty more secure than other A/B testing platforms?
AB Tasty has eliminated the use of JavaScript’s eval() function, reducing the risk of code injection and making our platform fully compatible with strict Content Security Policies (CSPs). We also use strong encryption, access controls, and regular security audits.
How does AB Tasty compare to other A/B testing tools like Optimizely or VWO in terms of security?
Unlike some competitors, AB Tasty does not require exceptions for unsafe code execution, making it easier to deploy in secure environments and reducing risk.
On November 13th, AB Tasty brought together over 40 customers, partners, and digital experience leaders in New York City for Experience Next NYC. The event was a celebration of innovation, community, and the future of experimentation and personalization.
Spotlight on Customer Presentations
A highlight of the day was hearing directly from three standout brands, each sharing their unique journeys and results with AB Tasty:
1. Five Below Five Below showcased how they leverage experimentation to better understand their Gen Z audience and optimize the digital experience for a fast-growing, trend-driven retail environment. Their team shared practical examples of how rapid testing and data-driven decision-making have helped them stay ahead of shifting customer preferences and deliver engaging, relevant experiences at scale.
2. Physicians Mutual Physicians Mutual discussed how they leverage AB Tasty’s EmotionsAI to segment users by emotional engagement. This AI-driven tool analyzes user behavior to assign visitors to one of 10 emotional segments, enabling more precise targeting and personalization. By using EmotionsAI, Physicians Mutual can better understand and address the emotional needs of their customers, optimize digital journeys, and drive measurable results, all while maintaining compliance and trust in a regulated industry.
3. FootJoy The FootJoy team inspired the room with their story of achieving a 9 out of 10 test success rate, far above the industry average. With a small team and no prior testing background, they embraced AB Tasty’s AI-powered tools to ideate, execute, and report on experiments quickly and confidently. Their experience is a testament to how accessible, AI-driven experimentation can empower teams of any size to deliver outsized results
More Event Highlights
Hands-On Learning: Attendees got an exclusive first look at Wandz, AB Tasty’s newly acquired real-time adaptive customer experience platform. Wandz enables brands to deliver personalized experiences for the 90% of visitors who are anonymous or non-logged-in, adapting to every click, scroll, and session pattern in real time. With predictive AI, Wandz anticipates visitor intent within milliseconds, setting a new standard for personalization and helping brands achieve up to 15% revenue lift and 13.6% higher conversion rates. Showcasing how Wandz will close the gap for brands looking to engage every visitor, not just those with a known profile.
Meet Evi: The event also introduced Evi, AB Tasty’s new evidence-based marketing agent. Evi is more than just an AI assistant, it’s a suite of intelligent tools designed to empower your entire workflow, from idea generation to reporting. Evi helps teams move faster, test smarter, and turn data into strategy without the guesswork. With features like Evi Ideas, Evi Content, Evi Hypothesize, and Evi Analysis, teams can unlock data-backed inspiration, craft clear hypotheses, and deliver straightforward insights, all in one place. Early adopters are already seeing a 53% increase in campaigns launched and 33% more experiments created, making Evi a true sidekick for digital teams.
Looking Ahead
Experience Next NYC was more than just an event, it was a celebration of our global customer community’s drive to push the boundaries of digital experience. The energy and ideas shared are already translating into new collaborations and customer initiatives, and we’re excited to see how these connections will shape the future of experimentation and personalization together.
Missed the event or want to relive the highlights?
Watch the full recap video here
Stay tuned for more Experience Next events and opportunities to connect, learn, and innovate together!
For two decades, the digital playbook has been clear: get clicks. Whether you’re selling sneakers or flights, success has been a game of climbing search rankings, optimizing landing pages, and guiding users through a funnel you meticulously built on your own website. That predictable path from a Google search to your checkout page is now being fundamentally rerouted.
The era of AI-driven discovery is here. Tools like Perplexity, Google’s AI Overviews, and ChatGPT are shifting user behavior from searching to asking. Instead of a list of blue links, users get a direct answer, a curated summary, or a complete travel itinerary. Now, with AI models integrating “buy” functionality, the journey is being short-circuited entirely. The conversation itself is becoming the point of sale.
This isn’t just another channel to manage; it’s a paradigm shift that challenges the core assumptions of digital marketing. For e-commerce and travel brands, the question isn’t just “how do we adapt?” but “what are we adapting to?” The truth is, nobody has all the answers yet. What follows isn’t a playbook, because a playbook doesn’t exist. It’s a pragmatic look at the shifts we’re seeing, how we might start to measure this new world, and why a culture of experimentation has never been more critical.
The new reality: From search to answers
The fundamental change is the introduction of a powerful new middle layer between a user’s intent and a brand’s website. Large language models (LLMs) are becoming expert synthesizers. A user asking, “What are the best running shoes for marathon training under $150?” no longer gets ten articles to read. They get a direct, compiled answer listing three specific models with summarized reviews and maybe a link.
This is the great unbundling of the search results page. The user gets their answer without ever needing to visit multiple sites to compare and contrast. And with platforms like ChatGPT embedding purchasing capabilities, that final step—the transaction—can happen right there in the chat interface. The website, once the center of the customer journey, risks becoming a simple fulfillment endpoint or, in some cases, being skipped entirely.
E-Commerce impact: When the storefront shrinks to a chat window
For e-commerce brands, this shift feels personal. The product detail page (PDP) is sacred ground. It’s a carefully crafted space for storytelling, cross-sells, and brand building. When discovery and comparison happen inside an AI, that ground vanishes.
The immediate impacts are clear:
A drop in direct traffic: Fewer users will land directly on product or category pages, making it harder to guide them through a curated experience.
The conversion conundrum: If a sale is initiated in a chat and fulfilled on your site (or via an API), how do you attribute it? Traditional last-click models become obsolete.
Lost opportunities: The spontaneous cross-sell (“Customers also bought…”) or the carefully placed upsell becomes much harder when you don’t own the interface.
Success in this new ecosystem may hinge on a brand’s ability to be “AI-friendly.” This isn’t about keywords; it’s about data. The brands most likely to be recommended by an LLM will be those with impeccable, highly structured product data that the AI can easily parse and trust. Your product catalog becomes your new landing page.
Travel’s new tour guide: The AI agent
The travel industry is perhaps even more exposed to this disruption. An LLM is, in effect, the ultimate travel agent. A single prompt like, “Plan a 5-day family-friendly trip to Lisbon in May, staying near the city center with a budget of $2,000,” can generate a complete itinerary with hotel options, flight suggestions, and activity booking links.
Brands risk being reduced to a single line item in an AI-generated plan. The key challenges are:
Disintermediation: If the AI presents three hotel options that all meet the user’s criteria, the brand’s own marketing and website become secondary to the AI’s curation.
Data accuracy is everything: Travel is time-sensitive. An AI won’t recommend a hotel or flight if it can’t confidently access real-time availability, accurate pricing, and clear policies. Outdated or poorly structured data is a death sentence.
Commoditization: Without the ability to showcase a unique brand experience on their own site, hotels and airlines risk being chosen on price and basic features alone.
For travel, the path forward requires a radical focus on the quality and accessibility of data. Think rich, structured, and instantly available information that makes your offering the easiest and most reliable choice for an AI to recommend.
Attribution in the age of answers
So, how do we measure success when clicks and rankings no longer tell the whole story? This is where the uncertainty is most palpable. The new platforms are largely opaque, and a new set of metrics is still emerging.
The conversation is shifting from “how did they find our site?” to “are we part of the AI’s conversation?” Potential new measures might include:
Mentions and citations: Tracking how often your brand or products are cited as answers to relevant queries.
Branded query lift: An increase in users asking for your brand by name (“Find me Nike running shoes”) becomes a powerful indicator of success.
Referral attribution: As partnerships form, tracking referrals directly from AI platforms will be crucial, though likely limited to their chosen partners.
For now, tracking remains experimental, but some signals are becoming clearer. We can now see referral traffic from sources like chat.openai.com and perplexity.ai in analytics. However, traffic from Google’s AI Overviews is currently blended with traditional organic search, making it difficult to isolate. This means a complete picture is still impossible, requiring a combination of brand monitoring and deep analysis of the referral data we can get.
Can brands catch up? The case for test-and-learn
This new search paradigm is full of unknowns, but waiting for a settled playbook isn’t a strategy. The only viable posture is a disciplined, test-and-learn mindset. The goal is to make your brand as legible, authoritative, and accessible to AI as possible, preparing you for whatever comes next.
Potential strategies include:
Mastering structured data: Implementing comprehensive schema markup across your site is no longer optional. It’s the cost of entry.
Creating AI-friendly content: Develop clear, factual, and easily digestible content that directly answers common customer questions, making it prime material for an LLM to cite.
Investing in brand and loyalty: When users are overwhelmed with AI-curated choices, a trusted brand name becomes a powerful shortcut. Loyalty programs and excellent customer experiences will be more important than ever.
Exploring API integrations: For larger brands, pursuing direct API integrations with major chat platforms could be a way to ensure your inventory and data are seamlessly included in their results.
The honest truth is that this ecosystem is still being built, and the rules are changing in real-time. The brands best positioned to navigate this shift won’t be the ones who guess the future correctly, but those who build a culture of rapid experimentation. The only question is, what will you try next?
AB Tasty is thrilled to announce that Wandz.ai, the real-time adaptive customer experience platform, is officially joining our family. This partnership is more than just a handshake; it’s a leap forward for digital experience optimization everywhere, and it’s a game-changer for personalization.
Why Wandz? Because Digital Journeys Are Messy—and We’re Here for It
Everyone in the digital space knows that online journeys are rarely straightforward.
Visitors pop in mid-funnel, juggle multiple tabs, bounce between devices, and most of them remain anonymous. In fact, 90% of your website visitors don’t log in or leave a data trail. But here’s the twist: when brands adapt to these unpredictable behaviors, conversion rates can skyrocket.
Wandz was built for this challenge. Wandz technology captures real-time intent signals—every click, scroll, and session pattern—and instantly adapts the experience for each visitor, even if they’re flying under the radar. Think of it as a digital guide that knows what your customers want before they do. The key to personalizing for your visitors is also understanding your users’ intent. That’s how you’ll know what to offer them next.
What’s in It for You, Personally?
With Wandz joining AB Tasty, we’re not just adding new features—we’re unlocking a whole new level of personalization and engagement for your brand.
Real-Time, Adaptive Personalization: Wandz’s AI engine processes 3.7 trillion data points every month, serving 1.5 billion users worldwide, and generates predictions in just 20 milliseconds. That’s not just fast—it’s lightning fast.
Proven Results: Leading brands in the digital e-commerce space have already seen up to 15% revenue lift, 13.6% higher conversion rates, and 65% more content engagement thanks to Wandz’s adaptive approach.
Full Transparency: No black-box mysteries here. Every data point is transparent, accessible, and actionable—so your teams can make informed decisions with confidence.
The Power of Predictive and Generative AI—Now in Your Hands
By bringing Wandz’s AI into the AB Tasty platform, we’re giving you the tools to move from reactive to proactive. Imagine forecasting behaviors, segmenting intelligently, and building models tailored to your unique customer journeys—all while optimizing every touchpoint in real time.
With generative AI, you can create and refine personalized experiences at scale, ensuring every interaction feels relevant and unique. Predictive AI keeps fine-tuning performance and outcomes, so you’re always ahead of the curve.
What’s Next? Smarter, More Adaptive Experiences for All
This isn’t just an acquisition—it’s a partnership for the future. By combining Wandz’s talent and technology with AB Tasty’s global reach and culture of innovation, we’re setting a new standard for adaptive personalization and customer experience.
Soon we’ll be rolling out these new AI-powered capabilities across all our clients. Whether you’re in marketing, product, or development, you’ll have the tools to analyze, strategize, and optimize like never before—transforming simple interactions into adaptive experiences that drive real business results.
Welcome to the Next Chapter of Digital Experience
So, what’s new at AB Tasty? Only the most advanced, adaptive, and engaging digital experiences on the market. We can’t wait to see what you’ll create with the combined power of Wandz and AB Tasty.
Stay tuned—this is just the beginning. The future of customer experience is adaptive, and we’re excited to help you lead the way.
We’ve all been there. Sprawled on the couch, phone in hand, dreaming up the perfect getaway. You scroll through stunning destinations, compare flight times, and find a hotel that looks just right. It’s exciting. It’s inspiring. And then… you put your phone down, deciding you’ll book it later on your laptop.
Sound familiar? It’s a story playing out millions of times a day.
This jump from mobile browsing to desktop booking is more than just a common habit; it’s a multi-billion dollar friction point for the travel industry. Your customers are dreaming on the go, but they’re hesitating to commit. The good news? This isn’t a dead end. It’s an opportunity. It’s a chance to turn that hesitation into confidence and those browsers into bookers, right where they are.
Let’s dive into what’s holding mobile travel back and how your team can start building a better, more trusted experience. Because good things happen to those who change.
Discover our Travel Essentials Kit to unpack 10 game-changing strategies that turn your digital experience into the smoothest journey from search to check-in.
The mobile paradox in travel
The numbers tell a fascinating story. Mobile devices are the undisputed engine of discovery in the travel sector, driving the lion’s share of online traffic. According to industry analysis from Zoftify, mobile is responsible for approximately 60% of all visits to travel websites. Yet, research from TravelPerk shows that despite accounting for the majority of browsing sessions, mobile devices represent a much smaller fraction of actual sales, with an estimated 60% of all bookings still coming from desktops.
That’s a huge gap between interest and action. While users love the convenience of browsing on their phones, there’s a clear disconnect when it comes time to pull out a credit card. This isn’t just a missed opportunity; it’s a signal that the mobile experience isn’t meeting the moment. Customers are ready to be inspired on mobile, but they aren’t yet convinced it’s the best place to make a high-stakes purchase. The challenge for your team is to bridge that gap.
Trust and performance: barriers to mobile conversion
So, what’s causing this hesitation? It boils down to two critical factors: performance and trust. When you’re asking a user to spend hundreds or even thousands of dollars, their confidence in your platform has to be absolute.
The data reveals where those cracks appear. According to research from Quantum Metric, it’s no wonder consumers have trust issues: 59% have experienced slow performance, 49% have had payment failures, and 43% have dealt with app crashes. On top of that, 45% have encountered bugs, causing half of them to abandon what they were doing. Each hiccup erodes trust. It plants a seed of doubt that asks, “If the site can’t even load properly, can I trust it with my booking?” This feeling is backed by the numbers; data reported by Navan indicates that only 25% of consumers feel fully confident completing a travel booking on their mobile device. That’s the core of the challenge. It’s not about a lack of desire, it’s about a lack of confidence.
UX moves that boost confidence and usability
Building confidence doesn’t require a complete overhaul. It starts with smart, user-centric design choices that make the experience feel seamless and secure. Every smooth interaction is a small deposit in the user’s trust bank.
Here’s where your team can start making an immediate impact:
Respect the thumb-zone: We navigate our phones with our thumbs. Placing key calls-to-action (CTAs) and navigation elements at the bottom of the screen makes them easy to reach and reduces physical effort. It’s a small change that makes your app feel instantly more intuitive.
Simplify every form: No one enjoys typing on a small screen. Keep your forms lean by removing non-essential fields. Enable guest checkout to remove registration barriers, use progress indicators on multi-step forms, and provide inline validation so users can fix errors in real-time.
Keep essentials above the fold: When a user lands on a mobile page, the most critical information and the primary CTA should be immediately visible without scrolling. This orients them instantly and shows them exactly what to do next.
Experimentation ideas specific to travel mobile
Understanding best practices is one thing. Knowing what works for your audience is where the real progress happens. This is where you move from fixing problems to finding your unique better. It’s time to embrace a mindset of “trial and better.”
Here are a few bold ideas to get your team started:
Test your navigation
Is a traditional hamburger menu really the best fit, or would a bottom navigation bar increase engagement with key sections? Run an A/B test to see which style helps your users find what they need faster.
Dial up the trust signals
Experiment with the placement and design of security badges and payment logos (Visa, PayPal, etc.) in your checkout flow. Does adding a “Secure Checkout” lock icon next to the “Book Now” button increase conversions? Let the data decide.
Optimize for perceived performance.
A content-heavy page doesn’t have to be a slow page. Experiment with technical solutions like progressive image loading for your visuals. This method loads a placeholder image first that sharpens as it fully loads, delivering content to the user more quickly. This improves actual load performance and keeps users engaged from the moment they land.
Run micro-experiments on upsells
The mobile booking flow is a delicate dance. A poorly timed upsell for baggage or a seat upgrade can feel disruptive. Test different triggers for these offers. Do they perform better when presented right after flight selection, or on the final confirmation page?
Bridging browsing to booking, from insight to action
The gap between mobile traffic and mobile conversion isn’t an unsolvable problem. It’s a series of smaller challenges waiting for creative solutions. By using benchmark data, you can identify your users’ biggest pain points and prioritize where to focus your efforts first.
Building a culture of iterative testing is the key. Small wins add up, creating a powerful momentum that continuously improves the user experience. As you monitor shifts in your mobile conversion rates and order values, you’re not just watching metrics. You’re seeing the direct result of your team’s courage to try, learn, and find what’s better.
How can you get your website to deliver different messages that resonate with different users? That was the challenge faced by UK travel company, On The Beach. They asked AB Tasty to help them speak to their different customer segments, leveraging data-driven decision-making to get more beaches to more people.
Founded in 2004, On The Beach is one of the UK’s largest online package holiday retailers, serving more than 1.7 million customers every year. Known for their colorful and dynamic brand image, they’ve built their reputation on providing affordable, hassle-free beach holiday experiences at a wide range of destinations. They’ve also recently branched out into offering city breaks around the world.
By promoting transparency, flexibility, and excellent customer service, On The Beach has established itself as one of the most trusted brands in the UK travel industry. You can find out more in our case study On The Beach Tests the Water with Personalization.
Same website, different customer journeys
A common problem for online travel companies is that consumers often spend time comparing flights or hotels across different websites before making the final decision to book. This can involve multiple visits to a particular website before they are ready to buy. Our research shows this is the single biggest factor influencing consumers to leave a website without booking travel options.
On The Beach is no exception to this trend. They cater to a wide range of customers, and their website traffic is a mix of both new and returning users. To help convince visitors to remain on their website and book, they want to show their different customer segments that they understand their different needs.
To do this, they try to provide them with personalized messages at different stages of their buying journey. This in turn helps people get the information they need and progress towards checkout.
Different messaging strokes for different folks
Testing and experimentation are key to helping On The Beach find the right message for each of its different customer segments. As Conversion Rate Optimization Manager, Alex McClean, explains,
“Testing and experimentation is important to us for two reasons. One is to help us understand our customers and what they want to see on site. And two is to help us learn and understand what we want to be able to do for our customers to help them”.
One example of the A/B testing that On The Beach carries out is trialing different badges to recommend the same holiday destination to different website visitors. With AB Tasty’s help, they discovered that new website visitors preferred holiday recommendations that were marked with a “Bestseller” badge. Returning users on the other hand, responded better to the same destination if it was marked with an “Our pick” badge. This was because they already trusted On The Beach to make holiday recommendations for them.
By testing these different messages, On The Beach was able to determine what message was right for what group. This led to a direct increase of more than 200 bookings on their website.
More tests and more of everything else
Initially, On The Beach started experimenting with some simple A/B tests based on content, product placement, and how different elements of their website performed. They then started to gradually increase the number of tests they did each month and bring other team members on board.
Now, On the Beach has developed a real culture of experimentation. It makes sure that all departments, including marketing, product development, and customer support have access to the latest testing information. And this greater level of involvement across the business also means that more hypotheses for tests come back to the marketing team.
This collaboration between teams has enabled On The Beach to make more data-driven decisions, successfully optimize different areas of its website, and continuously improve its customers’ user experience. The end result is greater customer satisfaction and increased growth for the business.
Alex McClean says,
“When we first started using AB Tasty, we were looking at rolling out five to ten tests a month. But now, as the business has advanced its testing capabilities and more people are getting involved, we’re rolling out 20-30 tests, with buy-in from the whole business”.
A helping hand from AB Tasty
For On the Beach, one of the major advantages of using AB Tasty’s experimentation optimization platform is being able to learn quickly and at scale. Using A/B testing, they can now make improvements to their website, iterate, and grow much more quickly than before. And this lets them provide visitors to their website a buying experience that speaks to all their customer segments.
Another key benefit for On The Beach is the support they receive from AB Tasty’s teams. Our responsive and knowledgeable support staff assist On The Beach in setting up tests, interpreting the results, and implementing optimizations on their website. We provide them with timely, personalized assistance, guiding them through the entire testing process and offering them expert advice on best practices.
But don’t just take our word for it. As On The Beach’s Alex McClean says, “For us, the real reason that we chose AB Tasty, and what differentiated AB Tasty from the competitors was the level of service that we were offered. We have complete access to developers, really attentive CSMs. It’s really beneficial to keep things moving when we don’t have to go to our development team.”
