AB Tasty is a complete tool for website and conversion rate optimization. We serve as your digital lab, equipped with everything you need to create experiments that will help you to better understand your users and customer journeys so that you can create the clearest and most engaging user experience possible, ensuring your website performs well and yields the maximum results.
In an era where privacy regulations tighten, browser restrictions escalate, and trust is hard-won, brands need more than great ideas to drive their digital experiments — they need full control over how their technologies behave.
That’s why AB Tasty is proud to introduce Domain Delegation, a groundbreaking feature designed to place independence, performance, and compliance at the heart of your experimentation strategy.
Why Domain Delegation changes the game
The digital landscape is constantly shifting at a fast pace. With evolving browser privacy policies (like ITP and ETP), widespread ad blockers, and stricter data regulations, third-party scripts are increasingly vulnerable — slowing down your site, triggering blockers, or worse, being outright rejected.
For enterprises operating under rigorous security standards, these challenges can make it nearly impossible to deploy tools like AB Tasty efficiently.
That’s where Domain Delegation steps in.
This powerful new feature allows you to serve the AB Tasty tag from a custom subdomain you control (e.g., abt.yourdomain.com), while AB Tasty takes care of the heavy lifting behind the scenes.
What you can do with Domain Delegation
Host the AB Tasty tag on your own subdomain (e.g., abt.brand.com)
Easily delegate DNS management to AB Tasty through an intuitive guided setup
Bypass blockers, improve load speed, and boost reliability
Deliver the tag under your own brand, reinforcing trust and compliance
Minimal technical effort, fully managed from the AB Tasty interface
What’s in It for You
✅ Higher tag reliability ⚡ Better site performance & Core Web Vitals 🔐 Stronger data governance & security posture 🤝 More brand trust with white-labeled tag delivery
Who benefits the most?
Highly regulated industries: Finance, healthcare, government
Privacy-first brands: Total data flow ownership
Tech teams optimizing performance and autonomy
Any organization battling browser or ad blockers
Why Now?
Privacy restrictions aren’t going away. Ad blockers aren’t easing up. With Domain Delegation, AB Tasty empowers you to take back control over your experimentation stack — ensuring you stay compliant, performant, and trusted.
This isn’t just a technical feature. It’s a strategic foundation for the next era of digital experimentation.
How It Works
Define your subdomain (e.g., abt.mybrand.com)
Follow the easy delegation flow in AB Tasty’s interface
Let us handle the rest (provisioning, certificates, delivery)
Your tag. Your domain. All powered by AB Tasty.
Domain Delegation Availability
Interested in Domain Delegation? Contact your AB Tasty Customer Success Manager to get started.
Every insight starts with a story, and every story deserves to be heard. But when your NPS® or CSAT campaigns generate thousands of responses, how do you turn all that feedback into real action, fast?
That’s why we created Feedback Copilot, the AI-powered assistant that transforms your NPS® or CSAT campaigns into actionable intelligence – instantly.
The problem collecting feedback: Too many voices, not enough time
Let’s face it: analyzing feedback is a nightmare. Even when users leave valuable insights in NPS campaigns, the manual work required to analyze hundreds (or thousands) of verbatim responses can paralyze teams. One client told us:
“We received 5,000 verbatim responses. That’s two weeks of manual work.”
And because it’s so time-consuming, teams either:
Underutilize feedback tools like NPS/CSAT
Or don’t act on the insights at all
The solution to overwhelming feedback? Feedback Copilot
Feedback Copilot was born from this pain point – combining the best of AI with our all-in-one experimentation platform. It’s available for free within AB Tasty, and automatically activated for NPS/CSAT campaigns with over 100 responses.
What our Feedback Copilot does:
Segments feedback by sentiment: Instantly separates positive and negative comments based on campaign scores.
Clusters similar comments into key themes: Groups feedback into topics like “price,” “delivery,” or “UX.”
Summarizes each theme: Provides a short description, confidence score, and sample comments for every theme.
Highlights what matters most: Surfaces the top 3 positive and top 3 negative drivers of satisfaction.
Exports labeled feedback: Download results for use in Excel, PowerPoint, and more.
And it does all this while respecting data privacy, using a self-hosted model (Hugging Face) instead of sending sensitive content to third-party LLMs.
What makes our Feedback Copilot unique?
Instant categorization of massive feedback volumes
Quantification of qualitative input – finally, your verbatim responses have numbers to back them
Integrated NPS/CSAT in your test workflows – measure why something works, not just if it does
Enterprise-grade privacy: Comments stay on AB Tasty’s infrastructure
Who benefits from Feedback Copilot?
CROs & Product Managers: Prioritize optimizations based on real user pain points.
UX & Research Teams: Detect trends and go beyond basic survey stats.
Marketing & Customer Success: Understand friction points before and after launches.
What’s next?
Early adopters already report major productivity gains – and they’re asking for more:
Direct A/B test ideas from negative themes
Verbatim-based segmentation for campaign targeting
Improved theme granularity for enterprise-scale campaigns
We’re just getting started. Feedback Copilot is not just a feature – it’s your co-pilot in delivering better, faster, and more human-centered product decisions.
In today’s digital landscape, data-driven choices are essential for staying competitive, with experimentation as a critical driver of innovation. To support this, we recently hosted a webinar with experts from Google Cloud and AdSwerve, focusing on how Google Analytics 4 (GA4) and BigQuery can enhance experimentation strategies. GA4 is essential for all marketing teams, providing advanced analytics that, when combined with BigQuery’s data consolidation capabilities, enables more effective testing, personalization, and digital optimization.
Meet the panel
Taige Eoff, Cloud Data AI Lead at Google, has been at Google for twelve years, leading data and AI initiatives for cloud marketing. Taige focuses on developing scalable solutions that support partners like AB Tasty and AdSwerve in optimizing digital experiences.
Alex Smolin, Senior Optimization Manager at AdSwerve, brings extensive experience in media, data, and technology. As a certified Google Premium Partner, AdSwerve provides data-driven brands with solutions ranging from A/B testing to advanced analytics.
Mary Kate, our roundtable host and Head of Growth Marketing for North America at AB Tasty, leads efforts to help companies create impactful digital experiences through AB Tasty’s suite of experimentation, personalization, product recommendations, and site search tools.
AB Tasty’s integration with GA4 & BigQuery
Connecting AB Tasty with GA4 gives marketing teams insights into visitor behavior through advanced analytics on CPA, conversion rate, bounce rate, SEO, and traffic. This integration allows teams to use data from either tool to measure the impact of experiments pre- and post-rollout, generating data-backed hypotheses and fostering innovation.
Google BigQuery, a fully managed cloud data warehouse solution, offers rapid data storage and analysis at scale. With its serverless, cost-effective structure, BigQuery allows businesses to analyze large datasets efficiently, making it easier to make well-informed decisions.
With Google BigQuery, users can effortlessly execute complex analytical SQL queries and leverage built-in machine-learning capabilities.
Why is data from GA4 foundational to any CRO program?
In experimentation, data is the catalyst that drives actionable insights. Data flows in from multiple sources, and businesses generate detailed reports by working with partners to integrate tracking and tagging. But the question then becomes: what comes next? That’s where experimentation enters. Using data from tools like GA4, teams can transform hypotheses into tests, uncovering which changes impact user engagement or conversions most effectively.
GA4’s role extends further by providing a consistent framework for testing across platforms. When integrated with BigQuery, GA4 allows teams to cross-reference test outcomes with other data points, revealing not just what worked but why it worked. As Alex noted, “We gather good data, run good tests, and then verify results across disparate sources like BigQuery to see if what we tested had the expected downstream impact.”
Data accessibility and agility are also important. Trends evolve quickly, with viral content or market shifts requiring rapid adaptability. “Having partners like Google, with all data in one place, and a platform like AB Tasty, where experiments can be quickly set up, is essential for staying competitive” Alex emphasized.
“Having partners like Google, with all data in one place, and a platform like AB Tasty, where experiments can be quickly set up, is essential for staying competitive.”
Alex Smolin, Senior Manager Optimization at Adswerve
How BigQuery powers scalable experimentation
With the growing volume of data, businesses need a way to consolidate and interpret it to drive impactful decisions. BigQuery, as Taige explained, is a robust cloud warehouse that streamlines data for meaningful insights, making it a key player in the experimentation ecosystem.
“Think of BigQuery as a filing cabinet for your organized data,” Taige noted. By consolidating disparate data sources, teams can create a unified view that informs testing and optimization efforts. Through this approach, tools like GA4 and BigQuery enable accurate decision-making that scales with the business. With BigQuery as the backbone, AB Tasty and AdSwerve can build on this structure to optimize user experiences through precise experimentation.
Beyond just data storage, BigQuery integrates with various Google Cloud tools and supports a wide range of use cases—from standard reporting to advanced machine learning. For marketers, this means fewer technical bottlenecks and quicker access to the data needed to stay agile. As Taige explained, “You may not need deep technical skills to access BigQuery’s benefits; the right partnerships and data structure can give you a powerful, accessible foundation.”
Leveraging BigQuery’s built-in AI and machine learning models
BigQuery offers an array of AI models for specific use cases—from translation and personalization to customer segmentation. These models add value by automating processes, such as localization or customer behavior prediction, allowing for smoother, more targeted marketing.
BigQuery’s flexibility means that companies can incorporate custom or third-party models, ensuring compatibility with a variety of AI solutions. This adaptability helps organizations innovate and iterate on experimentation programs, expanding what they can achieve with data.
Simplifying data access for marketing efficiency
For marketing teams, BigQuery’s role as a centralized data hub allows seamless data consolidation from platforms like Google Ads, Salesforce, and GA4. This integration ensures that marketers aren’t slowed down by fragmented data sources, freeing them to focus on insights and execution. As Taige highlighted, “The peace of mind that BigQuery provides comes from knowing that all data is consolidated and accessible, allowing teams to be nimble and creative.”
With BigQuery, marketers can view performance metrics, analyze customer journeys, and refine strategies—all within a unified environment. This lets teams optimize campaigns in real time as new data insights emerge.
Next-Generation capabilities enabled by Google Cloud
Looking ahead, digital is paving the way for more advanced experimentation capabilities. The conversation shifts to AI and machine learning, bringing new opportunities for personalization and optimization. As Mary Kate pointed out, while AI-driven insights can revolutionize customer experiences, many brands are still years away from realizing the full potential of these tools.
True value will come not from adopting every new tool but from understanding the foundational data supporting AI and asking the right questions about how these technologies can serve customer needs. Taige added, “If you don’t have a data strategy, you won’t have an AI strategy.” While AI amplifies data power, it requires organized, high-quality data to work effectively.
By consolidating and centralizing data through BigQuery, teams gain real-time insights and can make informed decisions. This data foundation enables the current wave of omnichannel strategies and sets the stage for future AI applications. Businesses that adopt this holistic approach—consolidating data, optimizing channels, and preparing teams for AI—will unlock new experimentation opportunities and drive impactful customer experiences.
With GA4 and BigQuery, businesses have the tools to streamline data consolidation and power next-generation experimentation. Ready to join your data and experimentation? Discover how AB Tasty can help bring data-driven optimization to life.
For e-commerce success, added revenue from existing customers can be more efficient than constantly pursuing new ones. Returning buyers are a vital piece of this strategy. We recently sat down with industry experts to discuss how optimizing customer experiences can drive upselling and cross-selling opportunities. They shared practical approaches for boosting average order value (AOV) while nurturing customer loyalty and retention.
Our speakers, each experts in testing, optimization, and conversion rate, provided insights into how brands can increase revenue through personalized, thoughtful customer engagement.
Meet the experts
Nicole Story: Co-Founder & Director at Hookflash Analytics, leading experimentation in testing, optimization, and personalization.
Gerred Blyth: Chief Product Officer at Giftory, with a background in interaction design and e-commerce.
In this article, we’ll explore actionable strategies from the webinar to help you personalize to existing customers and drive growth through upselling and cross-selling—not just new customer acquisition.
1. Optimizing the cart for upsells
Upselling at the cart and checkout stages can significantly increase AOV, but it requires a carefully planned approach. As Colette Carlson explains: “Before implementing anything, it’s crucial to understand how you’re going to measure success and ensure that your conversion rate is solid. When it comes to the cart and checkout process, if those aren’t optimized, adding upsell and cross-sell strategies will only introduce more noise.” Shoppers who have reached the cart are already primed to convert, so it’s important not to disrupt their momentum with irrelevant or poorly timed offers.
“Before implementing anything, it’s crucial to understand how you’re going to measure success and ensure that your conversion rate is solid. When it comes to the cart and checkout process, if those aren’t optimized, adding upsell and cross-sell strategies will only introduce more noise.”
Collette Carlson, Director of Optimization at Astound Digital
Coordination with internal teams is also important when designing upsell strategies. For instance, if an upsell is introduced at checkout, the process should be seamless – will the original product be automatically removed from the cart if the customer selects an upgrade, or will they need to make the changes manually? Likewise, if you’re offering a bundle or cross-sell, is your system prepared to handle it without disrupting the customer experience?
Effective upsell offers are relevant to the customer’s purchase. Suggesting complementary items or upgrades can boost AOV, as 80%of consumers are more likely to complete their purchase with brands offering personalized experiences. From upsell testing experience at Giftory, Jared advises against pushing unrelated or overly expensive items, which can confuse or deter customers altogether.
Using product recommendation algorithms can streamline upselling. Automating this process ensures that customers receive relevant suggestions without the need for manual curation, creating a smoother experience for your team and the customer. AB Tasty’s product recommendation engine allows upsells based on several criteria, including most recent products, associated products, similar or more expensive items, complementary items, and top promotions.
2. Strategic product recommendations for cross-selling
To effectively cross-sell, brands must identify the right moment in the customer journey. If you offer relevant products at key points without disrupting the experience, similar to upselling. But first, establishing cross-selling metrics can lead to stronger effectiveness.
The primary metrics will vary depending on what you’re testing—whether it’s an algorithm change, a new carousel design, or a different recommendation format. There are some essential KPIs to consider:
Engagement: Track how often customers interact with cross-sell offers, such as clicks or add-to-basket rates.
Conversion rate: Measure how many customers who engage with offers complete their purchases.
Average order value (AOV): Gauge how effectively cross-sell strategies are increasing the total order value.
Items per order: Monitor if cross-sell efforts lead to additional products being added to the cart.
Overall revenue: This ultimate metric reflects the total impact of your cross-sell strategy.
Once these metrics are in place, refine your strategy by determining where cross-sell offers should appear. For example, adding a cross-sell option in the mini cart or as a pop-up at checkout can add complexity, so testing can help avoid disrupting the customer experience.
Testing cross-sell algorithms in action
Nicole Story shared a valuable example of testing product recommendation carousels. Inspired by Amazon’s success, many brands rushed to implement carousels on their websites, but forgot the importance of context. Placing multiple carousels on the homepage often leads to irrelevant suggestions and a poor experience.
Nicole’s team tested various algorithms by tailoring product recommendations to the customer’s journey. On product detail pages (PDPs), carousels that showed “related product suggestions” outperformed those with generic recommendations. The tests revealed that adjusting algorithms based on context and customer behavior was far more effective than placing standard carousels throughout the site.
As Nicole explains: “Simply introducing product recommendations and checking that box off the roadmap isn’t going to deliver real value. The key is continuous optimization and discovering what works across the entire customer journey—that’s where the real value lies.”
“Simply introducing product recommendations and checking that box off the roadmap isn’t going to deliver real value. The key is continuous optimization and discovering what works across the entire customer journey—that’s where the real value lies.”
Nicole Storey, Co-Founder & Director at Hookflash Analytics
Relevance is everything
Cross-sell strategies must be highly relevant to what the customer is already doing. As Gerred Blyth from Giftory mentioned, “We have high expectations as customers and irrelevant offers can break that trust.” Customers expect brands to know their preferences and behaviors, so it’s important that recommendations feel personalized and timely.
3. Experimentation and testing for long-term loyalty and CLV
Continuous experimentation is critical for building long-term customer loyalty and increasing customer lifetime value (CLV). Instead of relying on a single strategy, brands should constantly test and improve their approach. Colette points out that starting by analyzing existing order data can uncover natural cross-sell patterns. This provides valuable insights into which products are frequently purchased together.
For first-time visitors, bombarding them with upsell offers might backfire. Instead, let them become familiar with your brand and key products before introducing additional offers. In contrast, repeat customers may be more open to cross-sells that align with their previous purchases.
Upselling with product recommendations
According to our data a customized UX can boost revenue and increase basket size by up to 10%. Product recommendations can be seen as a form of personalization and, as our panel pointed out in the webinar, experimenting with different formats—such as carousels, quizzes, or other interactive tools—can help identify what resonates with your audience and drives engagement.
We use AI to analyze visitors’ site interactions and purchase behavior, delivering targeted recommendations, each with a specific goal. This means you can better understand which products to offer, to whom and when during the customer journey:
Product Page: Guide users to explore more products or categories.
Last Seen Products: Help users quickly resume their browsing.
Add to Cart: Encourage users to add complementary items to their basket.
Cart Page: Suggest additional items to increase order value.
Homepage: Showcase personalized content and help users navigate the site.
Our panel also discussed how different types of algorithms are necessary depending on your vertical. You can divide your algorithms into three distinct types and choose how you prioritize:
Convert: These recommendations would offer top sellers, top trending products, top converting products, top reviewed products etc.
Upsell: This could suggest most recent products viewed, associated products, similar products, compatible products etc.
Personalize: This could suggest last visited products, last bought products, user affinity or similar or associated to cart products
If you work for a beauty site, customers will replenish their favorite products, whereas home and decor might recommend accessories or similar products. While personalization drives relevance, maintaining control over the recommendation process means you can speak directly to your customer’s needs.
Giftory: fostering loyalty with timely engagement
Giftory is beginning to focus on lifetime customer value. Their approach involves using cross-sell and upsell strategies similar to a CRM initiative, introducing customers to a broader range of products both during and after their purchase. They gather data on why customers buy gifts, such as birthdays or anniversaries, and use that information to send timely product recommendations in the future.
By reaching out to customers at the right moment, such as 11 months after an anniversary purchase, Giftory can re-engage them with relevant offers without overwhelming them with constant promotions. This creates a personalized experience that encourages long-term loyalty and repeat business.
4. Subscription models for upsell and retention
Offering subscription products to upsell can improve both immediate revenue and CLV. The challenge is to find the right balance: How can you encourage customers to subscribe without overwhelming them, while also ensuring the offer feels relevant and valuable over time?
Before launching a subscription model, look at your data to understand customer behavior. Consider the difference between a one-time purchaser and a subscriber. While offering a small discount for subscribing may lower the initial AOV, the long-term benefits of recurring revenue from a loyal subscriber can make up for it.
Testing and data-driven strategy
Launching a subscription model requires more than just adding an upsell feature—it involves a data-informed approach. Starting small with a minimum viable product (MVP) allows you to test how customers respond and fine-tune the offering. Metrics like renewal rates, engagement, and overall CLV will help guide decisions about whether to scale the program.
As Gerred advises: “Walk before you run. Start with the first test—an MVP. It doesn’t have to be the final version you’ll roll out, but that initial test will help you understand the value and prove the benefits. From there, you can evolve and continuously improve. It’s easy to feel overwhelmed when you hear about advanced strategies and algorithms, but you don’t need to get there all at once.”
“Walk before you run. Start with the first test—an MVP. It doesn’t have to be the final version you’ll roll out, but that initial test will help you understand the value and prove the benefits. From there, you can evolve and continuously improve. It’s easy to feel overwhelmed when you hear about advanced strategies and algorithms, but you don’t need to get there all at once.”
Gerred Blyth, Chief Product Officer at Giftory
Offering personalized options, such as different subscription tiers or flexible renewal cadences (monthly, bi-monthly, quarterly), can make the experience more appealing to a wider range of customers. Testing, refining, and adapting based on customer feedback will ensure that the model evolves in a way that meets both business goals and customer expectations.
Wrapping up
Just as you approach CRO with care and precision, cross-selling and upselling require a high level of attention.
Upselling and cross-selling don’t have to be complex when you have the right tools and the right strategy. If you want the expert’s opinion – watch the webinar below:
In today’s mobile-first world, where smartphones dominate more than half of global web traffic, optimizing for mobile has never been more crucial. Mobile usage surpassed desktop in the US in 2022 and in the UK in 2023, signaling a clear shift in consumer behavior. Brands are now urged to design with mobile in mind first, adapting for desktop as needed, rather than the reverse. This shift may seem daunting for teams, but it’s a necessary evolution to meet the expectations of today’s users.
Whether your customers are researching products or making purchases, their mobile experience can make or break their journey with your brand. While it’s clear that more shopping is done on mobile devices than on desktop, the real question remains: how significant is mobile shopping overall? Today’s mobile-savvy consumer isn’t just using their device for convenience, but to blend their in-store and online shopping into one seamless experience. In fact, nearly 80% of shoppers globally use their smartphones to browse a retailer’s website while shopping in-store, and 74% use the store’s app. However, only 33% of consumers prefer making purchases on their phones, with 49% reporting a smoother experience on desktop or tablet. This highlights just how important it is for brands to enhance their mobile offerings for a seamless experience across all devices.
To delve into the complexities of mobile optimization Mary Kate, AB Tasty’s Head of Growth Marketing for North America, teamed up with Allie Tkachenko, a UI/UX Strategist at WPromote, for a webinar on mastering mobile. AB Tasty’s platform enables brands to deliver personalized customer experiences, while Wpromote helps design and optimize engaging web experiences that convert. They emphasize a key message: mobile optimization isn’t just about resizing for a smaller screen – it’s about creating an intuitive, seamless journey that aligns with today’s mobile-first consumer’s behaviors and expectations.
It’s critical that mobile websites excel in areas like speed, navigation, and user-friendliness. Let’s dig into three actionable strategies from the webinar to help your brand stay ahead and deliver an improved mobile experience for your customers.
1. Maximizing limited space
One of the biggest challenges in mobile design is maximizing limited screen space without overwhelming users. The key is to keep crucial content above the fold—on mobile, this means placing essential elements like navigation bars, CTAs, and product highlights in a prominent position, visible without scrolling. This is particularly important on search landing pages, the homepage, and other high-traffic areas. A well-organized and streamlined navigation system that helps users quickly find what they need can lead to higher engagement and reduced bounce rates.
While desktops offer ample space to break down navigation into detailed categories, mobile design requires a more simplified structure due to space constraints. Consider grouping categories under broader buckets like “Top Categories” or similar, allowing users to easily explore the site without feeling overwhelmed by too many options. Another key strategy is leveraging responsible design, such as implementing sticky navigation bars or menus that stay visible as users scroll. This approach, widely adopted across industries, ensures easy access to important links and minimizes the effort required to navigate the site.
AB Tasty in action
The UX team at Clarins wanted to make their product more visible on their category pages. In the original layout, filtering and sorting functions were stacked, removing space from the second row of products appearing. After testing a column layout for the filtering and sorting menus, the team saw a significant improvement—bounce rates decreased, and clicks to products increased by 34%.
The “Thumb Zone” refers to the area of the screen that is easiest for users to reach with their thumbs, typically the lower portion of the screen. Since most users interact with their phones one-handed, placing critical CTAs, buttons, and interactive elements within this zone is important for accessibility and ease of use.
Consider this: a navigation bar that starts at the top of the page but shifts responsively to the bottom as the user scrolls. This keeps it in an expected spot initially, avoiding any disruption to the user’s flow, and then moves it to a more reachable area as they continue browsing.
Another thing to keep in mind is sizing. Whether it’s buttons, images, form fields, or menu links, the size of these elements plays a huge role in usability. You can’t just shrink them to save space—you have to ensure they’re “tappable” so users can easily interact. While reachability is key, think about what doesn’t need to be within reach, like informational banners or logos. You can place those outside the thumb zone, saving prime space for interactive elements.
Brands that prioritize the thumb zone in their mobile designs see improved user engagement and lower frustration levels. This small shift can make a significant difference in usability and customer satisfaction.
AB Tasty in action
The team at Club Med, a leading travel and hospitality brand, observed that their original mobile site displayed a navigation bar at the top of the page, which would disappear as users scrolled down. To increase user engagement with different category offerings, they created a variation of the mobile homepage with a sticky navigation bar which remained at the bottom of the screen while scrolling.
The results of the A/B test revealed a 12% increase in click rates, a 12% increase in access to the transaction funnel, and a 2% decrease in the bounce rate for users showing the variation with the sticky navigation bar. This approach effectively makes information more physically accessible.
Optimizing the thumb zone
Bottom Navigation
Sizing
Reachability
3. Improving processes
Lengthy forms and cumbersome checkout processes are major obstacles to conversion in mobile digital experiences. Mobile users expect a seamless, fast journey, and frustration with complex forms often leads to abandoned carts. Streamlining these processes—especially form fills and checkouts—can reduce friction and improve conversions. We’ve all experienced the annoyance of having to redo a form, fearing progress might be lost, which can lead to users abandoning the process entirely. Key areas for optimization include simplifying checkout by offering guest checkout options and exploring one-click payment methods.
Search and product discovery also present unique challenges on mobile devices due to limited screen space. With condensed menus and site navigation, users often rely heavily on the search function. Optimizing your search results pages to help users quickly find specific products can drastically improve the user experience. The space constraints of mobile mean that every element, including search results, should guide users efficiently to what they’re looking for.
Lastly, page load speed plays a vital role in retaining users. A slow-loading site can deter users, leading them to abandon your site altogether. Reducing load times is crucial for keeping users engaged. Understanding your audience and continuously optimizing these processes will help ensure your site meets their needs and encourages conversions.
AB Tasty in action
Travel insurance company, DirectAsia, needed users to fill out a form to generate an insurance quote. The team observed that customers were not completing the forms as smoothly as expected. To address this, they implemented a variation in the test where bolded check marks appeared to validate each completed field. This change created a sense of progress for users as they navigated the form and alleviated any uncertainty about needing to go back to correct errors.
As a result of this test, DirectAsia achieved a 1.97% increase in quote generations and a 7.78% increase in transaction rates. By reassuring users throughout the form-filling process, DirectAsia successfully guided more customers through their quote generation form.
Optimizing mobile processes
Checkout
Search and discovery
Speed & image loading
Wrapping up
Mobile optimization is about much more than making your website look good on a smaller screen; it’s about crafting a seamless, user-friendly experience that enhances the customer journey. Whether you’re focusing on improving site speed, optimizing design for better accessibility, or streamlining complex processes, the suggestions above provide a solid foundation for mastering mobile optimization. By understanding the nuances of mobile behavior and catering to the needs of your users, your brand can create a frictionless experience that drives conversions and fosters customer loyalty.
Stay ahead in the mobile-first era by ensuring your website design and processes align with the expectations of today’s consumers. AB Tasty can help achieve this goal by providing innovative tools and data-driven testing to enhance your mobile strategy. As mobile usage continues to grow, so does the importance of providing a smooth, engaging, and conversion-focused experience.
If you want to get all the details. – watch the webinar below.
When it comes to optimizing your website for better performance, experimentation is key to uncovering the areas that will make the biggest impact. Every page and interaction is an opportunity to boost conversions, streamline the user experience, and reduce friction. But with so many possibilities, where do you start testing? What are the key hotspots on your website that can deliver quick wins and meaningful improvements?
Finding the right areas to test can be tricky, but knowing where to focus is important for getting results quickly. That’s why we’ve identified five key questions to help guide your experimentation efforts:
Is the content relevant to “me” (the visitor)?
Is the next action clear?
What may be distracting?
What could cause anxiety?
What is creating urgency?
By addressing these questions, you can start identifying the areas on your website—what we call Experimentation Hotspots—that offer the most potential for improvement. In this guide, we’ll explore these important areas, from the homepage to the checkout funnel, that are ideal for testing. Whether you’re optimizing for mobile, app, or desktop, these tips will help you kickstart your experimentation journey and start making impactful changes.
1. Homepage
The homepage is where you show people who you are and what your brand means. It serves as the gateway to various sections, from product categories to special offers, and is where users often return mid-session. Your homepage often sets the tone for the rest of the user’s journey, making optimization crucial.
One effective strategy is to experiment with the placement and labels of your call-to-action (CTA) buttons. Ensuring that users can quickly locate key actions, such as signing up or starting their shopping journey, can make a significant difference in engagement. Testing different CTA labels, such as “Explore Now” versus “Shop Now,” helps determine which drives the most conversions.
Additionally, it’s essential to test the layout and how content is presented. Category links, sales promotions, and other critical pieces of content should be placed in a way that draws attention without overwhelming users. Small adjustments—like changing the placement of sales banners or highlighting popular categories—can enhance the overall user experience and increase interaction with your content.
Areas to optimize
Header
Main Navigation
Categories
Content Pushes
Layout
AB Tasty customer success in action
The beauty brand, Lush, shifted its homepage messaging by testing two variations—one focused on self-care and another on gifting care packages. The version centered on gifting led to a 250% increase in engagement, showing how even minor tweaks in messaging can significantly improve user interactions.
2. Product listing page (PLP)
The product listing page (PLP) is where users begin narrowing down their choices. This page is most effective when visitors can scan and compare products easily, helping them make decisions more efficiently. While it might seem like there’s limited space to display information, there’s a lot of room for optimization that can smooth the path to purchase.
Testing how high-quality images, clear pricing, and CTAs like “Add to Cart” are displayed can improve product visibility. Adding reviews or badges like “Best Seller” can boost credibility and urgency. Experiment with hover effects to show additional details when users hover over a product.
Experiment with the placement and functionality of filters to make product discovery easier. Test whether a sidebar, dropdown, or dynamic filtering works best for your users. Simple adjustments to category labels or sorting options, like by price or ratings, can enhance the shopping experience.
Areas to optimize
Product Card Information (e.g., imagery, price, CTA)
Reviews and Badges
Hover States
Filters and Sorting Options
Headers/Banners
AB Tasty customer success in action
Clarins UK reintroduced the wishlist functionality to their PLP and PDP. This allowed users to save their favorite products for later, which led to a 1.54% increase in basket page views. It also increased the number of transactions by 1.83%, showing the value of small user experience enhancements.
It’s also important to consider the differences between mobile and desktop experiences when testing. A solution that works well on desktop might not perform the same on mobile due to varying user behaviors. Despite a long-standing focus on mobile optimization, it’s still common to overlook these differences.
3. Product display page (PDP)
The product description page (PDP) is the centerpiece of any e-commerce experience and at the center of a customer journey. Many retailers are now asking, “How can we make the PDP feel more like a landing page?” With more customers arriving directly on PDPs from social media, rather than the homepage, the PDP often serves as the first touchpoint. This makes it even more crucial, as it’s where customers ultimately decide whether or not to make a purchase. Reducing friction and providing clear, relevant information are essential to driving conversions at this stage.
Testing how high-quality images, clear pricing, and CTAs like “Add to Cart” are displayed can improve product visibility. Adding reviews or badges like “Best Seller” can boost credibility and urgency. Experiment with hover effects to show additional details when users hover over a product.
Experiment with the placement and functionality of filters to make product discovery easier. Test whether a sidebar, dropdown, or dynamic filtering works best for your users. Simple adjustments to category labels or sorting options, like by price or ratings, can enhance the shopping experience.
Key areas to experiment with
Product Imagery and Descriptions: Ensure your images and descriptions are high-quality, answering common questions about the product (e.g., size, fit, material).
Call-to-Action Buttons: Test the wording, design, and placement of your CTAs to find what drives the most conversions.
Additional areas to optimize
Page Layout (order of information)
Product Images and Videos
Shipping & Return Messaging
Cross-Sell Items
Call to Actions
Reviews and User-Generated Content
Experience for New Visitors
Mobile vs Desktop Experience
Add-to-Cart Behavior & Mini Cart
AB Tasty customer success in action
French beauty product L’Occitane implemented a social proof widget on its PDP, showing how many people had recently viewed a product. The first variation displayed “X people are viewing this product,” while the second version said “Selling Fast.” This simple tweak created urgency and reassurance, leading to a 5.76% increase in transaction rate and a 3.36% increase in revenue per visitor.
Social proof taps into user psychology—when customers see that many others are viewing or purchasing a product, it creates a sense of urgency and reduces anxiety, helping to drive sales.
4. Checkout funnel
The checkout process is where the real magic (or frustration) happens. It’s the final step in the customer journey, and even small obstacles here can lead to abandoned carts. Simplifying the checkout funnel can make the difference between completing a sale and losing a customer at the last moment. This stage is not just about speeding up the process—it’s also about building trust and ensuring customers feel confident in their purchase.
Optimizing your checkout involves refining everything from the mini basket to the confirmation page, making the experience as seamless as possible.
Testing opportunities
Product Imagery and Descriptions: Ensure your images and descriptions are high-quality, answering common questions about the product (e.g., size, fit, material).
Call-to-Action Buttons: Test the wording, design, and placement of your CTAs to find what drives the most conversions.
Additional areas to optimize
Basket Summary
Step Progression
Login and Guest Checkout Options
Account Creation
Form Field Labels & Microcopy
Payment Methods
Coupon Field Visibility & Design
Cross-Sells & Product Recommendations
Confirmation Page
AB Tasty customer success in action
Calvin Klein ran a test where they displayed a “last viewed” carousel on the empty cart page after a user removed all items from their cart. This reminded users of previously viewed products, reducing the bounce rate by 4.19% and increasing orders by 7.5%. By preventing the cart from feeling empty and offering product recommendations, the brand successfully retained potential customers and increased conversions.
5. Personalization
Personalization is key to creating a unique and engaging experience for each visitor. In an online shopping survey, 88% of respondents rated personalized product recommendations as helpful when asked to score them on a scale of 1-5 (with 5 being very helpful and 1 not helpful). By understanding user behavior and segmenting audiences, you can deliver tailored content that enhances user satisfaction and drives higher conversions. Personalization not only improves the shopping experience, but also fosters customer loyalty, encouraging repeat purchases over time.
Start with
Behavioral Targeting: Personalize content based on how users interact with your site. For example, use data on past clicks or scrolling behavior to serve more relevant offers.
Abandoned Cart Reminders: Encourage users to complete their purchase by reminding them of items left behind.
AB Tasty customer success in action
French fashion retailer, Maje created a personalized campaign for users who abandoned their carts. When those users returned to the site, they were shown the products they had previously left behind, making it easy for them to continue shopping. This strategy resulted in an 11% increase on click rate on the pop up, with a significant portion of those users completing their purchases.
In addition, AB Tasty tools like Emotions AI can help recognize emotional triggers or online body language (e.g., scroll behavior or clicks) to tailor content even further, providing users with the most relevant experiences.
Wrapping up
From optimizing your homepage to personalizing the user experience, every aspect of your website offers an opportunity for growth. Even small tests—like changing the placement of a carousel or adjusting CTA wording—can lead to significant results.
By regularly experimenting and optimizing key hotspots on your site, you can improve user engagement, reduce friction, and ultimately increase conversions. Remember, every site is different, so test frequently and use data to inform decisions. This approach helps ensure that you’re continually refining your site for optimal success.
If you want to get all the details. – watch the webinar below.
This guest blog was written by Kit Heighway, Director of Optimization, at Daydot, a digital agency that specializes in crafting exceptional experiences to drive measurable revenue growth. They are experts in Conversion Rate Optimization, Performance UX Design, and Customer Lifecycle Optimization for Subscription, eCommerce, and Non-Profit brands.
Let’s imagine you are relaxing at home after a long day’s work, when the doorbell rings. You jump up, eager to see what’s waiting behind the door – a new clothing item, a cooking kit, or perhaps a treat for your pet? The excitement is real, and it’s all thanks to your recent subscription box sign-up.
Subscription boxes have quickly become consumer’s favorite way to shop from brands they love. In fact, the global subscription box market is projected to exceed $75 billion by 2025.* With so many brands wanting to get in on the action, the key question is: how can you participate in a way that resonates with your audience? A/B testing subscription features is a great starting place.
The Daydot team dive into the subscription box world and try out 5 of the most popular subscription purchase journeys. In the article below, we share what features we loved the most, giving you our round-up of the best features to test on your digital subscription journey.
The subscription journeys reviewed: Abel and Cole, Bloom and Wild, Gin-box, Dear-bump, Bella & Duke, Perky Blenders, Butternut Box.
5 Subscription Trends to Test
1. Combining expected USPs with what makes your business different
In 2024 certain features have become standard expectations in the world of subscription services. Phrases like “Free delivery,” “Home delivery,” or “Cancel anytime” are no longer points of differentiation – they’re baseline consumer expectations. However, these essentials still matter, and this is where the savvier brands really stand out, by mixing their unique personality with those expected USPs.
Take Abel & Cole, for example. As the leader in organic products, they don’t settle for the dull “Free delivery to your door.” Instead, they integrate their brand personality into the message with “Get your ethical food delivery dropped to your door.” It’s a small tweak, but it adds a layer of authenticity and makes a difference in standing out.
Abel and Cole: “Get your ethical food delivery dropped to your door”
Test ideas:
Expected USP copy improvements
2. Cancelation reassurance throughout the user journey
Subscription cancellation will happen but how you handle it can make or break the customer experience. A recent study shows that over 25% of consumers prioritize easy, penalty-free cancellations when choosing a service.* People value the assurance that they can leave without any hassle.
The best brands excel in this area by offering clear, upfront reassurance about cancellation. From the product listing page, where they confidently state, “There’s no commitment – you can skip or cancel at any time,” to the basket page with a gentle reminder, “Delivered weekly, but you can cancel anytime.” Right before the final step, they reinforce the message: “Count on us for reliable weekly delivery, with the flexibility to skip, pause, or cancel anytime.”
By making cancellation easy and transparent, these brands turn a potentially stressful decision into another positive touchpoint, helping customers feel in control from start to finish.
Test ideas:
Homepage cancelation reassurance
Product listing/details cancelation reassurance
Basket cancelation reassurance
Checkout cancelation reassurance
Cancelation messaging tone formal vs friend
3. Showcasing how subscriptions could fit into users’ real lives
For physical subscriptions, the key is to make them feel tangible and exciting in the early stages of the purchase journey. In a digital world getting something tangible is a huge draw.
How can you showcase that experience on a website without sending samples?
Butternut Box achieves this with a fun, heartwarming video on their landing page. It walks potential users through the excitement of the box arriving at the door, the thrill of unboxing, and of course, a happy dog enjoying their treat. It’s more than just a video—it’s a mini-experience that brings the product to life.
And Abel & Cole? They’re leading the way again, by adding recipe ideas and videos directly on the product page. It’s far more engaging than a simple image gallery, sparking users’ imaginations and making their experience more immersive.
Test ideas:
An unboxing or arrival video
A social media feed showing real users enjoying your subscription
Previews of activities you can do with the subscription (like recipe guides, or dog games)
4. Remembering that users don’t just subscribe for themselves
It’s easy to overlook that many users aren’t subscribing for their own needs – they’re often gifting a subscription to someone special. Whether it’s for a child heading off to college or a new colleague at work, recurring subscription businesses miss this opportunity by sticking to a one-size-fits-all approach.
Perky Blenders, however, has mastered the art of gifting by offering flexible three, six, and twelve-month subscription options for their premium, freshly roasted coffee.
Test ideas:
Gifting subscription journey
Business gifting journey
Personalization based on gift giving intent (supporting a friend, new home, starting uni, new parent etc)
5. Not assuming that subscriptions will last forever
Consumers want subscriptions to be as hassle-free as possible. Hidden or complicated cancellation processes can be a major turnoff.
Surprisingly, more businesses aren’t testing fixed, limited-time prepaid subscriptions. Some customers don’t want to commit to an ongoing plan, no matter how easy cancellation is. Offering a set subscription period could disrupt the subscription box journey in a big way.
Bloom & Wild are ahead of the curve here, letting customers pick between three, six, and twelve month subscriptions without any automatic renewal.
Test ideas:
3 month fixed term subscription
6 month fixed term subscription
X time fixed term subscription (reflecting a particular life-stage your product may be purchased for)
Wrapping up
These five innovative strategies are helping subscription leaders enhance their customer journeys, boost conversions and drive revenue growth. Now is a good time to evaluate your own subscription flow and consider integrating some of these ideas. But don’t just copy and paste – Remember, about 80-90% of digital ideas flop because they weren’t tested first. That’s why Experimentation is essential before implementing. It allows you to identify what clicks with your users and ensures that you invest in features that deliver results, rather than relying on assumptions.
If your organization is having trouble successfully running A/B tests in areas of your site where customers are going through the purchase flow, the issue may be due to Single Page Applications (SPAs) on your site. As customers move through the process, your A/B testing tool might not recognize their progress in an SPA environment.
There is enormous value in A/B testing critical areas of the web experience that are often operating in SPA environments, such as an eCommerce checkout.
This guest blog post was written by Jason Boal, The Principal Analyst & Optimization Strategist at 33 Sticks – a leading American analytics agency. Let’s address this common issue and uncover ways to overcome this to unlock hidden revenue on your site.
1. What is a SPA and how can I tell if the web experience uses one?
In a Single-Page Application (SPA) environment, content is loaded dynamically without requiring a full page refresh or reload. User interactions occur on a single page, with new content being loaded as the user navigates. Gmail is a prime example of an SPA. At a high level, an SPA functions similarly to a standard client-server interaction, but the key difference lies in what is returned to the browser.
To determine if you are operating in a Single-Page Application (SPA) environment on your site, pay attention to whether the page reloads as you interact online. If you see the page load indicator—such as the spinning icon in the browser tab (in Chrome)—it means the page is reloading, and you are likely using a traditional multi-page application (MPA).
Many websites are hybrids, meaning that only certain sections, like the checkout process, function as an SPA. To find out which parts of your site are SPAs, you can ask your development team for clarification.
2. Does my testing tool work within SPA environments and what do I do if it doesn’t?
Visual editors are becoming extremely popular in the A/B testing space, for many reasons. One is that Marketers are developing and launching more tests compared to the DEV team. If your testing tool has a difficult time loading and testing content in the visual editor, the reason could be that the tool is either not equipped or not set up to properly handle SPAs. This often happens in a secure checkout flow, where the customer is required to step through items like shipping address, billing, etc. The page you are attempting to A/B test on will not properly load in the visual editor and you will receive an error message.
FIGURE 1 – VISUAL EDITOR SPA ERROR MESSAGE
Ask your vendor if your testing tool can detect changes in the DOM and if it has a mechanism to look for timing.
Here are two challenges that some A/B testing tools face:
Visual Editors: Some A/B testing tools rely on the initial page load to determine what content to modify. These tools may struggle when content needs to change without a page reload. For example, if your test content is on page 3 of your site’s checkout flow, which is an SPA, the tool might not detect the need to inject content changes because there are no page loads as users navigate through the checkout flow.
Timing: As content on the page changes, it can be tricky for an A/B testing tool to insert test variations at the right moment. Variations can’t be applied before the content starts loading, but waiting until the content has fully loaded can result in users seeing the content change, a phenomenon known as “flicker.”
AB Tasty has extensive experience creating A/B tests in Single-Page Application (SPA) environments. We recommend implementing a delay on our tag’s execution so that it only triggers when the page is fully ready. This is achieved using a proprietary locking mechanism. This is just one example of how AB Tasty stands out in the A/B testing industry.
3. How do I take it to the next level?
Once you’ve unlocked A/B testing in SPAs, it is time to brainstorm testing ideas and develop a strategic roadmap to uncover ways to increase revenue for your organization. Here are a few ideas to help you jump-start that process!
Test various methods of updating cart quantity.
Test product detail page functions such as color variant selection methods.
Test buy box functions such as stock amount and store information.
Test different shipping messages based on cart value.
Test reordering flow steps.
Test navigation patterns or menu structures to optimize user flow within the SPA.
A/B test various UI/UX elements like buttons, forms, or interactive features specific to your SPA.
Test personalization strategies based on user behavior and interactions within the SPA.
Key Takeaways
There is enormous value in A/B testing critical areas of the web experience that are often operating in SPA environments, such as an eCommerce checkout flow. This is usually the last stage of any digital customer journey and vital to get right.
Determine whether or not your site leverages SPAs anywhere on your site.
Dig into your testing tool to ensure it can properly load test content changes with SPA environments.
Understand what other AB testing tools are out there and how they handle SPAs.
Develop an optimization roadmap based on your new knowledge!
Shiva Manjunath shares how debunking best practices, embracing failure, and fostering a culture of learning can elevate experimentation to new heights.
In this episode of The 1000 Experiments Club, guest host and AB Tasty’s Head of Growth Marketing UK, John Hughes, sat down with Shiva Manjunath, Senior Web Product Manager of CRO at Motive and Host of the podcast From A to B. Shiva’s journey through roles at Gartner, Norwegian Cruise Line, Speero, Edible, and now Motive, has made him a passionate advocate for the transformative power of experimentation.
During their conversation, Shiva discussed the pitfalls of following “best practices” blindly, the importance of creating an environment where failure is seen as a step toward success, and how companies can truly build a culture of experimentation.
Here are some of the key takeaways.
The myth of ‘Best Practices’
Too often, the so-called experimentation best practices become a checkbox exercise, rather than a thoughtful strategy.
“If you’re focused on best practices, you’re likely missing the point of true optimization,” Shiva notes.
He recounted a situation at Gartner where simplifying a form—typically hailed as a best practice—actually led to a sharp drop in conversions. His point? Understanding user motivation and context is far more important than relying on one-size-fits-all rules. It’s this deeper, more nuanced approach to experimentation that drives real results.
“If what you believe is this best practice checklist nonsense, all CRO is just a checklist of tasks to do on your site. And that’s so incorrect,” Shiva emphasized, urging practitioners to move beyond surface-level tactics and truly understand their audience.
Embracing failure in experimentation
A major theme of the discussion was the pivotal role failure plays in the journey to success. Shiva was candid about his early experiments, admitting that many didn’t go as planned. But these “failures” were crucial stepping stones in his development.
“My first ten tests were all terrible. They all sucked,” Shiva admitted, underscoring that even the most seasoned experts start with mistakes. He stressed that organizations must create an environment where employees can experiment freely, learn from their mistakes, and continue to improve.
“If you’re penalized for running a losing test, you’re not in a culture of experimentation,” Shiva insists.
Organizations that punish failure are stifling innovation. Instead, Shiva advocates for an environment where employees can test, learn, and iterate without fear. “The idea that you have the flexibility to discuss failures and focus on, ‘Well, I ran this test. It lost. Now, what do we do next?’—that’s a culture of experimentation.”
Scaling experimentation maturity
Shiva also explored the varying levels of experimentation maturity within organizations. Many companies claim to have a “culture of experimentation,” but few truly practice it at scale. Shiva emphasized the importance of making experimentation accessible to everyone in the organization, not just a select few.
Reflecting on the loss of Google Optimize, Shiva acknowledged its role as a gateway into the world of experimentation. “I got into experimentation through Google Optimize,” Shiva recalled, recognizing the tool’s importance in lowering the barrier to entry for newcomers. He urged companies to lower barriers to entry and enable more people to engage with experimentation, thereby fostering a more mature and widespread culture of testing.
The role of curiosity and data in experimentation
Another critical point Shiva raised was the importance of curiosity in experimentation. He believes that genuine curiosity drives the desire to ask “why” and dig deeper into user behavior, which is essential for effective experimentation.
“If you’re not genuinely curious about the why behind many things, I don’t know if experimentation is the field for you,” Shiva stated, underscoring curiosity as a crucial soft skill in the field.
Shiva also highlighted the foundational role of being data-driven in any experimentation strategy. However, he cautioned that having data isn’t enough—it must be effectively used to drive decisions.
“If you’re in a business setting and the business looks at your program and this is zero test wins, right? And then after two years, they would rightfully say ‘is this the way it’s supposed to go?’” Shiva remarked, pointing out that data-driven decisions are key to sustaining a culture of experimentation.
What else can you learn from our conversation with Shiva Manjunath?
Why it’s crucial to critically evaluate industry buzzwords and ensure they align with real practices.
How true personalization in experimentation goes beyond just adding a user’s name.
The need for thorough analysis to genuinely support data-driven decisions.
Shiva’s take on the future of experimentation after Google Optimize and how companies can adapt.
About Shiva Manjunath
Shiva Manjunath is the Senior Web Product Manager of CRO at Motive and Host of the podcast From A to B. His insatiable curiosity about user behavior and deep passion for digital marketing have made him a standout in the world of experimentation. With experience at top companies like Gartner, Norwegian Cruise Line, and Edible, Shiva is dedicated to demystifying CRO and pushing the boundaries of what’s possible in the field.
About 1,000 Experiments Club
The 1,000 Experiments Club is an AB Tasty-produced podcast hosted by Marylin Montoya, AB Tasty CMO. Join Marylin and the Marketing team as they sit down with the most knowledgeable experts in the world of experimentation to uncover their insights on what it takes to build and run successful experimentation programs.
A/B testing is often seen as the magic bullet for improving e-commerce performance. Many believe that small tweaks—like changing the color of a “Buy Now” button—will significantly boost conversion rates. However, A/B testing is much more complex.
Random changes without a well-thought-out plan often lead to neutral or even negative results, leaving you frustrated and wondering if your efforts were wasted.
Success in A/B testing doesn’t have to be defined solely by immediate KPI improvements. Instead, by shifting your focus from short-term gains to long-term learnings, you can turn every test into a powerful tool for driving sustained business growth.
This guest blog was written by Trevor Aneson, Vice President Customer Experience at 85Sixty.com, a leading digital agency specializing in data-driven marketing solutions, e-commerce optimization, and customer experience enhancement. In this blog, we’ll show you how to design A/B tests that consistently deliver value by uncovering the deeper insights that fuel continuous improvement.
Rethinking A/B Testing: It’s Not Just About the Outcome
Many people believe that an A/B test must directly improve core e-commerce KPIs like conversion rates, average order value (AOV), or revenue per visitor (RPV) to be considered successful. This is often due to a combination of several factors:
1. Businesses face pressure to show immediate, tangible results, which shifts the focus toward quick wins rather than deeper learnings.
2. Success is typically measured using straightforward metrics that are easy to quantify and communicate to stakeholders.
3. There is a widespread misunderstanding that A/B testing is a one-size-fits-all solution, which can lead to unrealistic expectations.
However, this focus on short-term wins limits the potential of your A/B testing program. When a test fails to improve KPIs, you might be tempted to write it off as a failure and abandon further experimentation. However, this mindset can prevent you from discovering valuable insights about your users that could drive meaningful, long-term growth.
A Shift in Perspective: Testing for Learnings, Not Just Outcomes
To maximize the success and value of your A/B tests, it’s essential to shift from an outcome-focused approach to a learning-focused one.
Think of A/B testing not just as a way to achieve immediate gains but as a tool for gathering insights that will fuel your business’s growth over the long term.
The real power of A/B testing lies in the insights you gather about user behavior — insights that can inform decisions across your entire customer journey, from marketing campaigns to product design. When you test for learnings, every result — whether it moves your KPIs or not — provides you with actionable data to refine future strategies.
Let’s take a closer look at how this shift can transform your testing approach.
Outcome-Based Testing vs. Learning-Based Testing: A Practical Example
Consider a simple A/B test aimed at increasing the click-through rate (CTR) of a red call-to-action (CTA) button on your website. Your analytics show that blue CTA buttons tend to perform better, so you decide to test a color change.
Outcome-Based Approach
Your hypothesis might look something like this: “If we change the CTA button color from red to blue, the CTR will increase because blue buttons typically receive more clicks.”
In this scenario, you’ll judge the success of the test based on two possible outcomes:
1. Success: The blue button improves CTR, and you implement the change. 2. Failure: The blue button doesn’t improve CTR, and you abandon the test.
While this approach might give you a short-term boost in performance, it leaves you without any understanding of why the blue button worked (or didn’t). Was it really the color, or was it something else — like contrast with the background or user preferences — that drove the change?
Learning-Based Approach
Now let’s reframe this test with a focus on learnings. Instead of testing just two colors, you could test multiple button colors (e.g., red, blue, green, yellow) while also considering other factors like contrast with the page background.
Your new hypothesis might be: “The visibility of the CTA button, influenced by its contrast with the background, affects the CTR. We hypothesize that buttons with higher contrast will perform better across the board.”
By broadening the test, you’re not only testing for an immediate outcome but also gathering insights into how users respond to various visual elements. After running the test, you discover that buttons with higher contrast consistently perform better, regardless of color.
This insight can then be applied to other areas of your site, such as text visibility, image placement, or product page design.
Key Takeaway:
A learning-focused approach reveals deeper insights that can be leveraged far beyond the original test scenario. This shift turns every test into a stepping stone for future improvements.
How to Design Hypotheses That Deliver Valuable Learnings
Learning-focused A/B testing starts with designing better hypotheses. A well-crafted hypothesis doesn’t just predict an outcome—it seeks to understand the underlying reasons for user behavior and outlines how you’ll measure it.
Here’s how to design hypotheses that lead to more valuable insights: 1. Set Clear, Learning-Focused Goals
Rather than aiming only for KPI improvements, set objectives that prioritize learning. For example, instead of merely trying to increase conversions, focus on understanding which elements of the checkout process create friction for users.
By aligning your goals with broader business objectives, you ensure that every test contributes to long-term growth, not just immediate wins.
2. Craft Hypotheses That Explore User Behavior
A strong hypothesis is specific, measurable, and centered around understanding user behavior. Here’s a step-by-step guide to crafting one:
● Start with a Clear Objective: Define what you want to learn. For instance, “We want to understand which elements of the checkout process cause users to abandon their carts.”
● Identify the Variables: Determine the independent variable (what you change) and the dependent variable (what you measure). For example, the independent variable might be the number of form fields, while the dependent variable could be the checkout completion rate.
● Explain the Why: A learning-focused hypothesis should explore the “why” behind the user behavior. For example, “We hypothesize that removing fields with radio buttons will increase conversions because users find these fields confusing.”
3. Design Methodologies That Capture Deeper Insights
A robust methodology is crucial for gathering reliable data and drawing meaningful conclusions. Here’s how to structure your tests:
● Consider Multiple Variations: Testing multiple variations allows you to uncover broader insights. For instance, testing different combinations of form fields, layouts, or input types helps identify patterns in user behavior.
● Ensure Sufficient Sample Size & Duration: Use tools like an A/B test calculator to determine the sample size needed for statistical significance. Run your test long enough to gather meaningful data but avoid cutting it short based on preliminary results.
● Track Secondary Metrics: Go beyond your primary KPIs. Measure secondary metrics, such as time on page, engagement, or bounce rates, to gain a fuller understanding of how users interact with your site.
4. Apply Learnings Across the Customer Journey
Once you’ve gathered insights from your tests, it’s time to apply them across your entire customer journey. This is where learning-focused testing truly shines: the insights you gain can inform decisions across all touchpoints, from marketing to product development.
For example, if your tests reveal that users struggle with radio buttons during checkout, you can apply this insight to other forms across your site, such as email sign-ups, surveys, or account creation pages. By applying your learnings broadly, you unlock opportunities to optimize every aspect of the user experience.
5. Establish a Feedback Loop
Establish a feedback loop to ensure that these insights continuously inform your business strategy. Share your findings with cross-functional teams and regularly review how these insights can influence broader business objectives. This approach fosters a culture of experimentation and continuous improvement, where every department benefits from the insights gained through testing.
Conclusion: Every Test is a Win
When you shift your focus from short-term outcomes to long-term learnings, you transform your A/B testing program into a powerful engine for growth. Every
test—whether it results in immediate KPI gains or not—offers valuable insights that drive future strategy and improvement.
With AB Tasty’s platform, you can unlock the full potential of learning-focused testing. Our tools enable you to design tests that consistently deliver value, helping your business move toward sustainable, long-term success.
Ready to get started? Explore how AB Tasty’s tools can help you unlock the full potential of your A/B testing efforts. Embrace the power of learning, and you’ll find that every test is a win for your business.
