The Customer-Centric Data Series
A collection of insights from AB Tasty’s partners and internal experts on the importance of data in experience optimization.
A clearer picture with data
Our increasingly digital world has caused a shift in how businesses and their customers interact with each other. Customers today prioritize experience over other aspects of a brand. They have increasingly high expectations of their digital interactions and in turn, it challenges brands to continuously optimize their digital strategies to stay ahead of the competition. But how do you know if you are creating a memorable experience for your customers that keeps them coming back?
At AB Tasty, we believe experimentation is the key to unlocking a new level of the digital experience, and it all relies heavily on data. Every stage of the experimentation cycle is driven by data, from ideation, hypotheses, statistical management and analysis. Therefore, the way to set your company up for successful experimentation is to lead with data-based, customer-centric optimization strategies.
Hear from top partners and data experts at AB Tasty as they discuss how data helps brands better profile customers, understand their needs, and forge valuable emotional connections with them, as well as measure overall digital impact and build a data-based, customer-centric vision. Through their insights, you will learn about important topics such as the key metrics you should always measure, how to quickly identify areas of improvement, best practices for building loyalty, and more, helping you to build better experiences for your customers through data.
ZION and ZION
Analytics and Data Strategy at Zion & Zion
When I’m looking to get an experimentation program up and running, I start by establishing my business objectives. For e-commerce, it’s things like tracking conversions, average order value and revenue per visitor. Then, I look at the analytics and ask myself where are conversions not happening? Having this information will help you examine how your data-fueled experiments are moving the needle.
For experimentation ideas, I’ll look at the tools, see the actual customer feedback, leverage user panelists to give them choices and ask what they prefer. I’m always looking to hypothesize friction points because these will lead to the best ideas for testing. You’ll want as much data as possible at the most granular level possible for lead generation, gated content and micro-conversions. You’ll need this data not only to understand what to test, but also to run the tests themselves.
You’ll also need tools to support experimentation. A great way to boost your A/B testing program — and help drive more ROI — is with platforms like Contentsquare or FullStory. These offer more data on customer behavior and experience. Analytics platforms enable you to test multiple hypotheses by offering more educated guesses about ways to improve customer experience.
But the key thing — and I really want to emphasize this — is the data. You need three months of data before you begin testing if you want your results to be reliable, and you need to be sure that the data is accurate. Most businesses use Google Analytics and that’s a lot of data to handle and organize. A customer data platform represents a significant investment, but centralizing your data into one system is extremely useful for customer segmentation and detailed analysis. The sooner you can invest in a tool like a CDP, the better for a sustainable data architecture strategy.
Zion and Zion is a US-based full-service marketing agency whose work is rooted in research, data, creative, analytics, marketing technology and customer data platforms (CDPs). Operating at the intersection of strategy and creativity, their campaigns look as good as they perform.
If your competitors are testing new features and responding to user behavior, they’ll set a new standard to match. Neglect that and you risk being left behind.
Data and Analytics Director at Jellyfish
Senior Analytics & Optimization Manager at Jellyfish
Data is not only the center of optimization, but it’s also the basis of our continuous improvement methodology at Jellyfish. We use a three-step process: understand, correct and validate. We start by understanding user behavior. Then we define the hypothesis to correct the identified areas of improvement. Finally, we validate through A/B testing to determine what to implement or test again.
The core element connecting the entire process is data. KPIs will differ depending on the client, but the primary KPI will always be based on the “next step” for the user. For example, if a test is on a product page, the KPI is “add to cart.” Identify the metric that best explains the impact of the change you’re testing and you’ll have a higher chance of reaching statistical confidence.
When you’re running A/B tests with 50/50 split traffic, you’re only gathering half the story. That’s why you need statistical confidence to be sure that any bias in the sample is considered when examining the results. Statistical confidence also helps validate your hypothesis. It gives you the ability to decide whether an improvement has potential for implementation.
When you don’t reach statistical confidence, it’s actually worse than having a negative test outcome because you can’t be sure whether your optimization has even had an impact. And without any testing data there’s no analysis or improvement.
Looking at how successful companies maintain their online presence regardless of industry and benchmarking yourself against the best will ensure you stay on top of what users expect of your business. If every website is focused on continuous improvement, you need to keep improving too. If your competitors are testing new features and responding to user behavior, they’ll set a new standard to match. Neglect that and you risk being left behind. That’s why it’s so important to keep testing and improving and why CRO will never become obsolete.
Jellyfish is a full-service marketing and digital company based in the UK working to help clients unlock their full potential across all platforms from SEO to media, data analytics and insights.
Companies that build a culture of experimentation see exponential improvements.
Director of Behavioral Science at Conversion
What’s the role of emotional decision-making in marketing? People build mental models around their emotions, experiences and cultural associations. They think of some as “good” or “bad” and they tie emotion to them. The key for marketers is to understand which emotions resonate with which groups of people. This is where A/B testing can help you find clues as to what works and what doesn’t. Creating strong emotional connections is paramount, and through experimentation, you can create them throughout your sales funnel.
Emotion is particularly important for positioning both your brand and your products. The data you can glean from testing things like value propositions or copywriting is extremely valuable for successfully positioning a product. As customers move through their journeys, they’re going to have different emotions at different moments, including doubts. Give them signals to reassure them they’ve made the right decisions, and you’ll strengthen their loyalty to you.
When dabbling with emotions, an A/B test is the minimum. In fact, if you talk about A/B testing in an optimization win-or-lose mindset only, you’re not even close to its full potential. When making strategic decisions, experimentation can certainly help incrementally optimize things, but it can do bigger things, including helping you to make key decisions, better understand your customers, innovate and take risks. There are many possibilities that testing can do and teams need more than a CRO manager to unlock its full capabilities. Not enough people realize the power of advanced testing. Companies that build a culture of experimentation see exponential improvements and have a key competitive differentiator.
When you make an investment in experimentation, it’s going to help you go much further with your decision-making. I wouldn’t call any part of that “guesswork.” You’re simply combining qualitative with quantitative data to come up with better hypotheses for testing. It’s the heart of CXO.
Conversion, a global CRO agency, specializes in an evidence-driven approach to conversion rate optimization. They combine A/B testing, UX research and Personalization to improve clients’ websites and businesses.
A breadth of data–balanced between qualitative and quantitative gives the most accurate picture of user behavior.
UX Director Dentsu International
There is no single way to understand user behavior and how you analyze customer data will often depend on the specificities of your business problem. Behavioral science and neuroscience help us interpret emotions, and implicit response testing and eye behavior analysis can help develop more detailed user journey mapping. In many cases, a breadth of data-balanced between qualitative and quantitative gives the most accurate picture of user behavior.
User testing should be about observing. A user might say that they found a task easy, but I recommend you dig deeper to make sure their feedback is backed up by the data. Work with an understanding of how people’s minds work and your testing approach will be more robust. Experiences absolutely matter, but as researchers, being grounded in human psychology is important too.
Additionally, remember to take a holistic view when it comes to user behavior metrics: What areas within the business are being affected by friction on the website? It could be customer support, or perhaps it’s the checkout process or delivery. You need to examine this with testing because each element is vital to the digital experience to achieve superior/better UX.
When it comes to optimization, usability and revenue are essential, but they’re only the start of the story. Usability is the absolute baseline for which you should aim: your real focus should be on delighting your customers. Make that your North Star. Create emotionally resonant experiences, make your users happy, and the rest will take care of itself. There is data behind emotion and memory and brand perception, so play those cards in your favor and you can boost customer lifetime value too.
Dentsu International, based in Japan, is one of the largest global marketing and digital companies in the world. Dentsu International provides integrated solutions and client services across a range of disciplines including creative, media, commerce, data and technology.
A note on privacy
With great data comes great responsibility. There are growing privacy concerns around data, from far-reaching restrictions of GDPR in Europe and similar privacy laws in states like California, and it means businesses need to be careful when collecting data from customers. The worldwide nature of e-commerce makes compliance with these crucial, no matter where your company is based. Leading with transparency and accountability in this will not only help protect your company from legal concerns, but it will also help you generate trust in your brand amongst customers.
Ethical handling of customer data is mission-critical if brands are to succeed today. From big tech to retail, companies are taking an ethical and privacy-centered approach to data including consent-first data collection and customizable sharing options, both of which are great ways to give customers control over their data while also helping you set yourself up for being a trustworthy company to do business with.
Overcoming skepticism through building trust will continue to be vital to harnessing the data you need because as an article in the Harvard Business Review aptly put it, “Privacy is to the digital age what product safety was to the Industrial Age.”
Grow faster with AB Tasty
Get a custom walkthrough of the platform
Filip von Reiche
Senior VP MarTech Enablement at Wunderman Thompson
Initially, digital transformation was focused on giving brands an online presence, but over time, it has acquired new uses. Its latest purpose is to help brands create personalized experiences by generating the right content and flow for more relevant conversations with their customers, leading to more frequent conversions.
Personalization allows you to propose a solution that meets the consumer’s needs and knocks down barriers. So long as you respect the trust the consumer has placed in you by giving you their data, you should be testing and personalizing experiences for your consumers.
There are lots of different paths to take with personalization. From a consumer experience point of view, you need to test to discover how best to interact with them. How can you show them you want to help them without being intrusive? It might help to think of this in terms of a retail store experience by asking, “How would I, as a customer, want to be welcomed, assisted, guided?” Understanding this is the best way to start a personalization framework.
If a consumer has been engaging with a particular brand for a very long time, they’re going to expect a certain level of personalization. And it’s worth offering this level of personalization because the better you know your customers, the longer you can continue a conversation with them, resulting in loyalty, retention and hopefully, referrals.
This brings us to measuring impact and evaluating customer lifetime value (CLV). CDPs play an essential role in measuring CLV because they can combine data from dozens of sources to retrace a customer’s entire history of interactions with a brand. Using this data, you can measure how long you’ve been engaging with that customer, what the value of that engagement has been and what they’re interested in. This enables you to continue that conversation and tap into that emotional response that we’re all out there chasing.
Wunderman Thompson is a US-based creative agency and consultancy focused on inspiring growth with ambitious brands. They offer deep expertise across the entire customer journey, including communications, commerce, consultancy, CRM, CX, data, production and technology to provide end-to-end solutions for clients.
Customer lifetime value is a key metric that can help you look at data in a holistic way.
Managing Director at Realise Unlimited
Chief Analytics Officer at Realise Unlimited
Transforming a company to be truly data led is not an easy process. Key stakeholders need to commit 100% and teams need to be able to speak to one another openly. Conflicting objectives within teams can be a barrier for brands looking to maximize performance. The aim, when transitioning to a data-centric approach, is to access rich information about the customer and their context, to understand their needs and their behaviors. In an omnichannel approach, this is essential because consumers’ expectations will differ depending on their platform of choice.
Data centricity is a key enabler to personalization, but that’s not the full story. Personalization can be very difficult to get right because it means different things to different people. Once a brand starts to personalize, it needs to have the resources to create content for each segment and this, in turn, can lead to some very complicated workflows and messaging. It also requires a lot of data, coordinated thinking and fairly rigorous planning because once you’ve started automated personalization, you need to stay committed to see results.
A valuable metric in this whole approach is customer lifetime value (CLV). A company or its objectives may evolve over time, in which case the focus for CLV can also shift: from your company (how much profit it is making) to your customer (how they are behaving) and your staff (do they have the right tools to help them make decisions).
As companies face the ongoing challenges of doing business in a competitive and saturated market, being able to concentrate on adding value for customers through the brand’s activities will be essential to keeping them ahead of other players in their field. Brands that are less focused on transactional value with their marketing, that are not afraid to commit to data centricity and innovative experiences, are the most exciting and potentially enticing for consumers.
Realise Unlimited is a boutique analytical consultancy based in the UK. Founded in 2007 by consumer insights professionals, they specialize in helping organizations understand more about their customers with the delivery of their CRM or digital strategy.
The more you leverage experimentation and advanced forms of data science, the better your competitive advantage.
Director of Strategy at Cro Metrics
Open dialogue with the clients we serve is important. I like to ask executives whether they think someone coming to their website for the very first time has the same needs as someone coming for the fifth time. Of course, I could already tell them the answer is “no.” But think about “new” vs “returning” or the different avenues that might lead someone to your website — all of these people have very different contexts.
When it comes to understanding your customers’ needs, the more you leverage experimentation and advanced forms of data science, the better your competitive advantage. The reason? Customer needs are unique from one brand to another. An established brand can sell itself differently than a niche brand or a newcomer brand.
Customers are used to seamless online interactions, and they want to find products easily. And if a brand doesn’t deliver, they’ll switch to one that does.
The more granularly you understand your customers, the more it will seem to them that you’re able to read their minds; you’ll please them and win their loyalty and win in the marketplace. The businesses that are doing this — doing a lot of experiments and doing them well — are the businesses that win. It’s also how you win with your customers because if you’re better at meeting their needs, everyone wins.
Using disparate data sources isn’t just about prioritizing test ideas either, it’s also about better understanding test outcomes. There’s no one out there who can use data to predict which test ideas will win. If they could, everyone would have staggering win rates, but even when you look at Microsoft, Google and Netflix, only one in 10 ideas pans out. It’s how you tie those results back to other sources of customer data or analytics to piece together a narrative you think might be true and how you use those results to inform your next test idea that will help determine your success.
Cro Metrics is a marketing services agency in California serving scale-up companies with strategies to drive revenue growth through data-driven experimentation. They provide everything from individual channel support to fully-managed experiment-led marketing programs, and help businesses better understand their customers.
Founder and Managing Director at Conversionry
Imagine you’re an air traffic controller. You have multiple screens constantly feeding you information about airplanes. From these constant insights, you have what you need to make the decisions that ensure the planes land smoothly. The key is to pick out the critical data and make use of it. It’s the same process with testing. Gather insights, identify problems, brainstorm solutions and test, test, test to turn those problems into winning solutions for your business. You need to start running experiments because every day without a test is a day of lost revenue!
For those who have yet to establish an experimentation strategy, pinpointing customer problems is a good place to start. Look at your funnel, identify the drop-off step and observe the next page path to identify where users go. Getting a handle on what goes through the user’s mind and understanding their problems will lay the groundwork for testing. A/B testing can also serve as a constant flow of insights that enable you to make better decisions.
When it comes to insights, there are many ways to gather this data and many sources to consult. You can run heatmap analysis, look at session recordings, user polls, disseminate surveys to email databases and collect insights from user testing. All of this will be critical to establishing the full scope of your customers’ problems and their needs.
In terms of timing and flow, make sure to evaluate insights and customer problems coming in every month or every few weeks. Gather the data, make sense of it and identify the key problems to solve by aligning them with your overarching business goals. Once you get this rhythm down, you can start designing development testing, build the full experimentation roadmap and repeat the cycle to deliver ongoing improvements to your business and your bottom line.
Conversionry is an Australia-based conversion optimization and e-commerce experimentation agency. They partner with leading Australian and international brands to help them drive revenue from their web traffic by making data and evidence the priority and delivering best-in-class experimentation programs.
Vice President at Spiralyze
A customer-centric culture is a data-led business model, where both qualitative and quantitative data are essential, and experimentation plays a vital role. Quantitative data gives brilliant direction as it’s informed by how customers are interacting with your products. Qualitative data, on the other hand, stems from customer needs. Pairing this data provides invaluable information and opportunities for tailored experiences for your brand’s consumers.
To be a data-driven organization, you need an experimentation mindset, because a customer-centric culture is premised on innovation and constant change to meet customer needs. Website optimization and personalization are key drivers toward adopting a data-driven mindset, and building appetite for this within your company will help give you that competitive edge. In a data-driven organization, people at every level are empowered to contribute, because it’s data, not experience, that matters. No matter what role you play in an organization, you can see the benefit of your work, and each team member is rallied around serving your customer.
Data democratization is essential for building a customer-centric culture. Siloed data is one of the key obstacles to democratization, so ensuring data is not segmented to certain departments but available to everyone will better enable you to understand your customers. Equally important is a system for gathering, storing, interpreting and acting upon this data whenever possible, such as a CDP and an accompanying experimentation roadmap.
For those companies who understand the meaning of a data-driven culture, the dissemination of insights across the entire organization is what drives customer centricity. There’s also value in focus. Too much data equals clutter and distraction, and it can be difficult to process data into something actionable. Designing systems to capture the needed information at scale, while minimizing variance by individual interpretation is a key success factor for data-centric businesses today.
Spiralyze is an Atlanta-based, full-service marketing agency that helps companies grow by providing data-driven conversion rate optimization and experimentation services that make more website traffic convert into valuable leads and customers.
Your primary KPI is how you measure a test’s success, but your secondary KPI will help you determine if a test is worth implementing.
Chief Data Scientist at AB Tasty
Viewing data from experiments as absolute truth can lead to a risk of accepting errors in test results. It is crucial to validate your results with primary and secondary metrics and put win rates into context to root out anomalies. This will help you better understand how customer behaviors are impacted by your tests and shrink the chance of accepting biased data.
Take for example average order value. This metric can have misleading results or “extreme value” from a small group of your test audience purchasing high-value items. An experiment yielding a high AOV might lead you to incorrectly assume one variation is a winner, which is why it is important to validate tests with primary and secondary KPIs. Your primary KPI is how you will measure a test’s success, but your secondary KPI will help you determine if a test is worth implementing. If your primary KPI (AOV) shows an increase but your secondary KPI (conversion) drops, this is an indicator to perform the test again before pushing it live to better understand why AOV changed.
This same practice can be applied to understanding predictions from your experimentation tools. A prediction metric like gain probability that is based on only one number can be misleading without additional context. A test might have a 95% gain probability but with a 5% chance the test goes wrong, you need to understand how wrong it could go.
At AB Tasty, we use confidence intervals to provide best- and worst-case scenarios of how a test will perform and give context to your potential wins and losses. Having a range of possible results is helpful for understanding if a test is worth the potential gain or risk. For example, a worst-case increase of 1% conversion might seem small but for a low-effort test like changing copy, it might be worth the small win. On the other hand, a test that requires high production needs a worst-case scenario that will at least recoup the cost of implementation.
Measuring your tests with the right primary and secondary KPIs and putting your results into context will give you greater confidence in your data and strengthen your experimentation strategy.
AB Tasty is a global leader in experimentation, personalization and feature management solutions — enabling companies to validate ideas, while maximizing impact, minimizing risk and accelerating time to market. Businesses use AB Tasty to align their marketing, product and engineering teams and ensure increased efficiency, reduced costs and optimal end-user experiences.
Where data will take us
To build a customer-centric company, you need to prioritize customers above prospects in your day-to-day work. Often as companies grow, they forget this mindset in the interest of acquiring new customers. It’s crucial to keep your current customers at the core of what you do and continue to build on that foundation of trust. When satisfied customers go on to share their experiences, you’ll find prospects turn into new customers.
In the past 10 years, there has been a big push to collect as much data as possible from visitors, sometimes to the detriment of trust. But now we are in this new phase of taking that data and turning it into growth and value by interpreting and then acting on it. With data we know the right KPIs to use to bring value, and we can then go on to automate it to work for new customers.
Testing and experimentation have a huge part in this process. They set quantitative data that helps us identify problems apart from qualitative data that helps us find solutions. In order to build a customer-centric culture, we need to be able to adapt to new needs instead of assuming our solutions will apply the same way.
Faulty data can be the downfall of well-intentioned optimization strategies. Whether the data is biased or skewed by its audience, or our hypothesis is weak at the onset of an experiment, an overcollection of the wrong type of data can mean a major setback for teams. It’s impossible to analyze and if you can’t analyze you aren’t extracting insights. And this will leave you at a loss. Remember, the enemy of good data is too much data because you can’t orchestrate it.
No matter how consumer behaviors shift, having a strategy based on data first will help you make iterative changes with confidence instead of fear of the unknown.
Get a custom walkthrough of the platform