Join VP Marketing Marylin Montoya as she takes a deep dive into all things experimentation
Today, we’re handing over the mic to AB Tasty’s VP Marketing Marylin Montoya to kick off our new podcast series, “1,000 Experiments Club.”
At AB Tasty, we’re a bunch of product designers, software engineers and marketers (aka Magic Makers), working to build a culture of experimentation. We wanted to move beyond the high-level rhetoric of experimentation and look into the nitty gritty building blocks that go into running experimentation programs and digital experiences.
Enter: “1,000 Experiments Club,” the podcast that examines how you can successfully do experimentation at scale. Our podcast brings together a selection of the best and brightest leaders to uncover their insights on how to experiment and how to fail … successfully.
In each episode, Marylin sits down to interview our guests from tech giants, hyper-growth startups and consulting agencies — each with their own unique view on how they’ve made experimentation the bedrock of their growth strategies.
You’ll learn about why failing is part of the process, how to turn metrics into your trustworthy allies, how to adapt experimentation to your company size, and how to get management buy-in if you’re just starting out. Our podcast is for CRO experts, product managers, software engineers; there’s something for everyone, no matter where you fall on the maturity model of experimentation!
We are kicking things off with three episodes, each guest documenting their journey of where they went wrong, but also the triumphs they’ve picked up from decades of experimentation, optimization and product development.
He believes anyone can and should do experimentation
In the culture of experimentation, there’s no such thing as a “failed” experiment: Every test is an opportunity to learn and build toward newer and better ideas. So have a listen and subscribe to “1,000 Experiments Club” on Apple Podcasts, Spotify or wherever you get your podcasts.
Is experimentation for everyone? A resounding yes, says Jonny Longden. All you need are two ingredients: A strong desire and tenacity to implement it.
There’s a dangerous myth lurking around, and it’s the idea that you have to be a large organization to practice experimentation. But it’s actually the smaller companies and start-ups that need experimentation the most, says Jonny Longden of performance marketing agency Journey Further.
With over a decade of experience in conversion optimization and personalization, Jonny co-founded Journey Further to help clients embed experimentation into the heart of what they do. He currently leads the conversion division of the agency, which also focuses on PPC, SEO, PR — among other marketing specializations.
Any company that wants to unearth any sort of discovery should be using experimentation, especially start-ups who are in the explorative phase of their development. “Experimentation requires no size: It’s all about how you approach it,” Jonny shared with AB Tasty’s VP Marketing Marylin Montoya.
Here are a few of our favorite takeaways from our wide-ranging chat with Jonny.
.
The democratization of experimentation
People tend to see more experimentation teams and programs built at large-scale companies, but that doesn’t necessarily mean other companies of different sizes can’t dip their toes in the experimentation pool. Smaller companies and start-ups can equally benefit from this as long as they have the tenacity and capabilities to implement it.
You need to truly believe that without experimentation, your ideas won’t work, says Jonny. There are things that you think are going to work and yet they don’t. Conversely, there are many things that don’t seem like they work but actually end up having a positive impact. The only way to arrive at this conclusion is through experimentation.
Ultimately, the greatest discoveries (for example, space, travel, medicine, etc.) have come from a scientific methodology, which is just observation, hypothesis, testing and refinement. Approach experimentation with this mindset, and it’s anyone’s game.
Building the right roadmaps with product teams
Embedding experimentation into the front of the product development process is important, but yet most people don’t do it, says Jonny. From a pure business perspective, it’s about trying to de-risk development and prove the value of a change or feature before investing any more time, money and bandwidth.
Luckily, the agile methodology employed by many modern teams is similar to experimentation. Both rely on iterative customer collaboration and a cycle of rigorous research, quantitative and qualitative data collection, validation and iteration. The sweet spot is the collection of both quantitative and qualitative data — a good balance of feedback and volume.
The success of building a roadmap for an experimentation program comes down to understanding the organizational structure of a company or industry. In SaaS companies, experimentation is embedded into the product teams; for e-commerce businesses, experimentation fits better into the marketing side. Once you’ve determined the owner and objectives of the experimentation, you’ll need to understand whether you can effectively roll out the testing and have the right processes in place to implement results of a test.
Experimentation is, ultimately, innovation
The more you experiment, the more you drive value. Experimentation at scale enables people to learn and build more tests based on these learnings. Don’t use testing to only identify winners because there’s much more knowledge to be gained from the failed tests. For example, you may only have 1 in 10 tests that work. The real value comes in the 9 lessons you’ve acquired, not just the 1 test that showed positive impact.
When you look at it through these lenses, you’ll realize that the post-test research and subsequent actions are vital: That’s where you’ll start to make more gains toward bigger innovation.
Jonny calls this the snowball effect of experimentation. Experimentation is innovation — when done right. At the root, it’s about exploring and seeing how your customers respond. And as long as you’re learning from the results of your tests, you’ll be able to innovate faster precisely because you are building upon these lessons. That’s how you drive innovation that actually works.
What else can you learn from our conversation with Jonny Longden?
Moving from experimentation to validation
How to maintain creativity during experimentation
Using CRO to identify the right issues to tackle
The required building blocks to successful experimentation
About Jonny Longden
Jonny Longden leads the conversion division of Journey Further, a performance marketing agency specializing in PPC, SEO, PR, etc. Based in the United Kingdom, the part-agency, part-consultancy helps businesses become data-driven and build experimentation into their programs. Prior to that, Jonny dedicated over a decade in conversion optimization, experimentation and personalization, working with Sky, Visa, Nike, O2, Mvideo, Principal Hotels and Nokia.
About 1,000 Experiments Club
The 1,000 Experiments Club is an AB Tasty-produced podcast hosted by Marylin Montoya, VP of Marketing at AB Tasty. Join Marylin and the Marketing team as they sit down with the most knowledgeable experts in the world of experimentation to uncover their insights on what it takes to build and run successful experimentation programs.
Chad Sanderson breaks down the most successful types of experimentations based on company size and growth ambitions
For Chad Sanderson, head of product – data platform at Convoy, the role of data and experimentation are inextricably intertwined.
At Convoy, he oversees the end-to-end data platform team — which includes data engineering, machine learning, experimentation, data pipeline — among a multitude of other teams who are all in service of helping thousands of carriers ship freight more efficiently. The role has given him a broad overview of the process, from ideation, construction to execution.
As a result, Chad has had a front-row seat that most practitioners never do: The end-to-end process of experimentation from hypothesis, data definitions, analysis, reporting to year-end financials. Naturally, he had a few thoughts to share with AB Tasty’s VP Marketing Marylin Montoya in their conversation on the experimentation discipline and the complexities of identifying trustworthy metrics.
Introducing experimentation as a discipline
Experimentation, despite all of its accolades, is still relatively new. You’ll be hard pressed to find great collections of literature or an academic approach (although Ronny Kohavi has penned some thoughts on the subject matter). Furthermore, experimentation has not been considered a data science discipline, especially when compared to areas of machine learning or data warehousing.
While there are a few tips here and there available from blogs, you end up missing out on the deep technical knowledge and best practices of setting up a platform, building a metrics library and selecting the right metrics in a systematic way.
Chad attributes experimentation’s accessibility as a double-edged sword. A lot of companies have yet to apply the same rigor that they do to other data science-related fields because it’s easy to start from a marketing standpoint. But as the business grows, so does the maturity and the complexity of experimentation. That’s when the literature on platform creation and scaling is scant, leading to the field being undervalued and hard to recruit the right profiles.
When small-scale experimentation is your best bet
When you’re a massive-scale company — such as Microsoft or Google with different business units, data sources, technologies and operations — rolling out new features or changes is an incredibly risky endeavour, considering that fact that any mistake could impact millions of users. Imagine accidentally introducing a bug for Microsoft Word or PowerPoint: The impact on the bottom line would be detrimental.
The best way for these companies to experiment is with a cautious, small-scale approach. The aim is to focus on immediate action, catching things quickly in real time and rolling them back.
On the other hand, if you’re a startup in a hyper-growth stage, your approach will vastly differ. These smaller businesses typically have to show double-digit gains with every new feature rollout to their investors, meaning their actions are more so focused on proving the feature’s positive impact and the longevity of its success.
Make metrics your trustworthy allies
Every business will have very different metrics depending on what they’re looking for; it’s essential to define what you want before going down the path of experimentation and building your program.
One question you’ll need to ask yourself is: What do my decision-makers care about? What is leadership looking to achieve? This is the key to defining the right set of metrics that actually moves your business in the right direction. Chad recommends doing this by distinguishing your front-end and back-end metrics: the former is readily available, the latter not so much. Client-side metrics, what he refers to as front-end metrics, measure revenue per transaction. All metrics then lead to revenue, which in and of itself is not necessarily a bad thing, but that just means all your decisions are based on revenue growth and less on proving the scalability or winning impact of a feature.
Chad’s advice is to start with the measurement problems that you have, and from there, build out your experimentation culture, build out the system and lastly choose a platform.
What else can you learn from our conversation with Chad Sanderson?
Different experimentation needs for engineering and marketing
Building a culture of experimentation from top-down
The downside of scaling MVPs
Why marketers are flagbearers of experimentation
About Chad Sanderson
Chad Sanderson is an expert on digital experimentation and analysis at scale. He is a product manager, writer and public speaker, who has given lectures on topics such as advanced experimentation analysis, the statistics of digital experimentation, small-scale experimentation for small businesses and more. He previously worked as senior program manager for Microsoft’s AI platform. Prior to that, Chad worked for Subway’s experimentation team as a personalization manager.
About 1,000 Experiments Club
The 1,000 Experiments Club is an AB Tasty-produced podcast hosted by Marylin Montoya, VP of Marketing at AB Tasty. Join Marylin and the Marketing team as they sit down with the most knowledgeable experts in the world of experimentation to uncover their insights on what it takes to build and run successful experimentation programs.
One of the pioneers of experimentation shares a humbling reality check: Most ideas will fail (and it’s a good thing)
Few people have accumulated as much experience as Ronny Kohavi when it comes to experimentation. His work at tech giants such as Amazon, Microsoft and Airbnb — just to name a few — has laid the foundation of modern online experimentation.
Before the idea of “build fast, deploy often” took hold across tech companies, developers followed a waterfall model that saw fewer releases (sometimes every 2-3 years). The shortening of development cycles in the early 2000s thanks to the Agile methodology and an uptick in online experimentation created the perfect storm for a software development revolution ― and Ronny was at the center of it all.
AB Tasty’s VP Marketing Marylin Montoya set out to uncover the early days of experimentation with Ronny and why failure is actually a good thing. Here are some of the key takeaways from their conversation.
.
Progressive deployments as a safety net
A typical cycle of experimentation involves exposing the test to 50% of the population for an average of two weeks before a gradual release. But Ronny suggests coming at it from a different vantage point: Starting with a small audience of just 2% before ramping up to 50%. The slower ramp-up gives you the time to detect any egregious issues or a degradation in metric values in near real time.
In an experiment, we may focus on just two features, but we have a large set of guardrails that suggest we shouldn’t be degrading X, Y or Z. Statistical data that you’re collecting could also suggest that you’re impacting something you didn’t mean to. Hence, the usage of progressive deployments in which you can identify external factors and easily rollback your test.
It’s like if you’re cooling water: You may realize you’re changing the temperature, but it’s not until you reach 0ºC (32ºF) that ice forms. You suddenly realize that when you get to a certain point, something very big happens. So, deploying at a safe velocity and monitoring the results can lead to huge improvements.
Your great idea? It will most likely fail.
Nothing gives you a better reality check than experimentation at scale. Everyone thinks they’re doing the best stuff in the world until it’s in the hands of their users. That’s when the real feedback kicks in.
Over two-thirds of ideas actually fail to move the metrics that they were designed to improve — a statistic Ronny shares from his time at Microsoft, where he founded the experimentation platform team of more than 100 data scientists, developers and program managers.
Don’t be deterred, however. In the world of experimentation, failing is a good thing. Fail fast, pivot fast. Being able to realize that the direction you’re going in isn’t as promising as previously thought enables you to use those new findings to enrich your next actions.
At Airbnb, Ronny’s experimentation team deployed a lot of machine learning algorithms to improve search. Out of 250 ideas tested in controlled experiments, only 20 of them proved to have a positive impact on the key metrics — meaning over 90% of ideas failed to move the needle. On the flip side, however, the 20 ideas that did succeed in some form? Those resulted in a 6% improvement in booking conversion, worth hundreds of millions of dollars.
The starter kit to experimentation
It’s easier today to convince leadership to invest in experimentation because there are plenty of successful use cases out there. Ronny’s advice is to start with a team that has iteration capital. If you’re able to run more experiments and a certain percentage are pass/fail, this ability to try ideas is key.
Pick a scenario where you can easily integrate the experimentation process into the development cycle and then work your way on to more complex scenarios. The value of experimentation is clearer because deployments are happening more often. If you’re working in a team that deploys every six months, there’s not a lot of wiggle room because everyone has already invested their efforts into this idea that the feature cannot fail. Which, as Ronny pointed out earlier, has a low probability of success.
Is experimentation for every company? The short answer is no. A company has to have certain ingredients in order to unlock the value of experimentation. One ingredient you need is being in a domain where it’s easy to make changes, such as website services or software. A second ingredient is you need enough users. Once you have tens of thousands of users, you can start experimenting and doing it at scale. And lastly, make sure you have trustworthy results from which you are taking your decisions.
What else can you learn from our conversation with Ronny Kohavi?
How experimentation becomes central to your product build
Why experimentation is at the root of top tech companies
The role leaders play in evangelizing an experimentation culture
How to build an environment for true experimentation and trustworthy results
About Ronny Kohavi
Ronny Kohavi is an authority in experimentation, having worked on controlled experiments, machine learning, search, personalization and AI for nearly three decades. Ronny previously was vice president and technical fellow at Airbnb. Prior to that, Ronny led the Analysis and Experimentation at Microsoft’s Cloud and AI group and was the director of data mining and personalization at Amazon. Ronny has also co-authored “Trustworthy Online Controlled Experiments : A Practical Guide to A/B Testing.,” which is currently the #1 best-selling data-mining book on Amazon.
About 1,000 Experiments Club
The 1,000 Experiments Club is an AB Tasty-produced podcast hosted by Marylin Montoya, VP of Marketing at AB Tasty. Join Marylin and the Marketing team as they sit down with the most knowledgeable experts in the world of experimentation to uncover their insights on what it takes to build and run successful experimentation programs.
Our UK partner series continues with Andrew Furlong, Managing Director at REO.
In this interview, we asked him the following 3 questions:
At REO, you “believe digital experiences can always be better” – what does that mean?
Like with most things, there is always room for improvement. The growth of AB testing and Personalization is because brands and consumers believe and demand a better experience. So, what we mean is: no matter how good you think your digital experience is, it can always be improved. And we are here to help if you are not sure how!
What are you most proud of at REO?
This one is easy… the team! They are great to work with, they are challenging and speak their mind to help REO be the very best it can be. Having such a brilliant team pays off as shown with our latest client satisfaction score of 8.5/10.
Which ultimate tip for experience optimization do you have for our readers?
It is important to always have multiple streams of optimization running, I am not talking about concurrent tests, although where feasible that should be done. I mean having a fallback strategy so that if a test is delayed for reasons beyond your control, you can quickly pivot to a different part of the site for example.
About REO
REO is a digital experience agency. We are an eclectic mix of bright and creative thinkers, embracing the best of research, strategy, design and experimentation to solve our clients’ toughest challenges. We work across a variety of sectors, with companies such as Amazon, M&S, Tesco and Samsung. To fearlessly transform our clients’ businesses and reputations by evolving the Digital Experience for their customers. We achieve this through:
Our curious and relentless drive to gather insights that matter.
Our proactive mindset and forward thinking to deliver lasting value.
Our adventurous approach to adapt and learn quickly.
In an increasingly cutthroat digital age, standing out in your niche while meeting the exact needs of your consumers is essential to business growth and longevity.
By getting under the skin of your customers, you can tailor your messaging, applications, and touchpoints to meet their exact needs —that’s where eye tracking enters the mix.
Consider this for a moment:
You’re running a usability test on a product landing page for a new range of gym shoes. Your test subject, Nancy, browses the page and chooses a shiny new pair of gym shoes with ease. But, on the next page, there is a snag. She hesitates and eventually abandons her cart because the journey was confusing.
You take notes based on Nancy’s feedback and think about how you can improve your checkout journey. But, if you could view her movements— or see what she sees—you would have the power to make informed improvements that will ultimately increase conversions and drive more sales.
With eye tracking, you can. But while this widely-used sensor-based technology offers a deep glimpse into user browsing behavior, some industry experts believe that eye tracking is an unnecessary expense.
Like many platforms and digital innovations, with the right approach, eye tracking will give you the tools to offer your customers a seamless level of user experience (UX)—the kind that will increase loyalty while helping you boost your bottom line.
Here we explore the dynamics of eye tracking and explain why it could make an excellent investment for your business.
So, what is eye tracking?
Eye tracking is a type of sensor technology that gives a computer or mobile device the tools to understand and trace where a person is looking.
An eye tracker can detect the presence, attention and focus of a user while engaging with a specific app, touchpoint or website.
From a marketing perspective, eye tracking dates back to the 1980s, where it was used to test and measure the value of ads in print papers or magazines.
An effective alternative to lie detection-style techniques such as voice stress analysis and galvanic skin response (neither of which offer truly reliable results or data), eye tracking gave the advertisers of the day essential insights into which elements of a page people read as well as how long they spend engaging with specific pieces of content.
The popularity of the eye tracker rose over the years and the rapid evolution of digital technology paved the way for a wealth of innovative developments.
Now, eye tracking technology is able to offer deep-dive insights into user behavior and dynamic page as well as app design as well as offering intuitive tools that enhance the user journey for disabled people.
In the modern age, one of the most prominent features of eye tracking is a little something called Facial Expression Analysis (FEA).
Based on ‘points of fixation’—times during the user journey when someone stops and focuses long enough to process the content before them (commonly known as a ‘saccade’)—FEA technology helps marketers gauge the effectiveness of their page design and messaging.
But, how does this apply to business and why is it so useful? Let’s find out.
As a marketer or business owner, the more you understand your target audience, the more chance you have of creating a fluent and engaging customer journey across platforms.
As eye tracking provides a visual map of how your users engage with your website, landing pages, and mobile applications, you can identify strengths and weaknesses related to user experience (UX) and content placement.
An essential part of the consumer research process, eye tracking is a powerful medium as it taps into the fact that 95% of human decision-making (particularly online) is carried out sub-consciously.
By using eye tracker tools to trace navigational patterns, you can adopt your customers’ vision, uncovering information that will help you to make improvements that boost engagement, improve your customer experience (CX) offerings, and ultimately, accelerate the growth of your business.
From heat mapping to task-based usability tools, there are a wealth of eye tracking innovations available to businesses in today’s digital world.
Invest in the right eye tracking tool for your business and you will:
Understand what your target audience is looking at and for how long
Identify redundant or disruptive visuals or design elements
Document how users scan and interact with your web pages or apps
Gain a practical understanding of what works and what doesn’t
Prove the value of certain marketing strategies, techniques or campaigns
Continually improve and evolve your efforts in a landscape that is ever-changing
Eye tracking is an effective means of seeing through the lens of your customers. But, as powerful as it is, eye trackers alone are unlikely to give you a complete insight into the content that really sticks in the users’ mind.
To gain additional context on how to use your data to improve usability and drive engagement, eye tracking should be a pivotal part of your consumer research strategy rather than a sole means of information.
That said, if you use it the right way, eye tracking can help you understand your customers in ways that can give you an all-important edge on the competition.
How eye tracking can help you understand your customers
Using eye tracking to understand your customers on a deeper level boils down to adopting a cohesive mix of the right tools and techniques.
Eye tracking tools and software provide a visual representation of your users’ focus points—returning data based on:
Fixation points or saccades: information that can tell you how engaging or eye-catching particular elements or pieces of content on a webpage are to your customers.
Navigational patterns: by understanding common navigational patterns, you can see how people scan or interact with your page. This level of knowledge will give you the data you need to optimize your content and design for increased engagement and conversions.
Problematic elements As mentioned, an eye tracking test will return invaluable data based on any images, graphics, calls to action (CTA) or command buttons, informational content or design elements that hinder the user experience and prevent customers from either getting what they need from your page or carrying out a desired action (clicking through to a specific product page or signing up to an email newsletter, etc).
Automotive repair and wreckage company, Truckers Assist, conducted eye tracking tests to track the performance of its homepage.
This test showed that while the ‘NO FEES’ graphic (the red point on the image) was gaining a lot of attention, it wasn’t clickable. As a result, many users were focusing their attention in the wrong place, steering them away from more valuable information as a result.
To fix this glaring issue, Trucker Assist improved its homepage design, removing the ‘NO FEES’ banner and placing focus on its contact information and service search bar.
Conducting successful eye tracking testing takes consistency as well as a clear cut goal. Do you want to improve the user journey of your new mobile app? Are you looking to drive more revenue through a specific product page? Perhaps you’re trying to understand if your general messaging and branding is performing the way it should?
There are many actionable insights you can gain from eye tracking—and outlining your specific goals will give your tests or studies direction.
This hand-picked video offers practical advice and information to help you get started with eye tracking:
How eye tracking benefits UX optimization
88%of consumers are less likely to return to a website after a poor user experience. Today’s consumers expect a seamless level of UX from brands and businesses—anything less and you could see customer loyalty as well as sales drop through the floor.
Eye tracking and UX go hand in hand. Through eye tracking, you will gain access to objective and unbiased insights that will show you where improvements are necessary.
With eye trackers, you can drill down into a specific UI element (is it facilitating the right interactions or are your consumers missing it altogether?) to test whether it fits into the user journey while getting to the very root of any distracting, problematic or misleading page elements.
This perfect storm of on-page information will empower you to make very specific improvements to any app, web or landing page—enhancing its usability and performance significantly.
How eye tracking works in a nutshell
As a concept, a significant part of eye tracking is based in Fitts’ Law. Essentially, every visual object or element carries a certain amount of ‘weight’ and this determines the amount of attention as well as clicks it ultimately earns.
Concerning eye tracking and UX, Fitts’s Law is important because it can help you predict the amount of time taken to move the eyes or cursor to a specific target.
Armed with this information, you can establish a visual hierarchy and optimize your webpages or applications to ensure consumers can connect with the right functions or information at the right times within their journey.
To get your eye tracking tests off to the best possible start, giving you users clearcut instructions while ensuring good lighting and consistent positioning is essential. Doing so will give you reliable data, as detailed in this infographic from IMOTIONS:
Essential eye tracking methods & techniques
There are thousands of eye tracking tools and countless ways of approaching this most powerful approach to user testing.
To guide you along the right path, here we’re going to explore the most essential eye tracking methods and techniques.
Heat maps
A branch of eye tracking, a heat map is a dynamic tool that offers a definitive visual representation of where users focus their attention and how they navigate your website based on their on-page interactions.
Heat mapping platforms provide color-coded data to give an indication of the areas of a website or mobile page users are interacting with the most.
As you can see from the image above, the red spots show the areas where users focus their attention most while the lighter colors are the areas with the lowest engagements.
Heat mapping technology also serves concrete data based on how much particular buttons or links are clicked by users on a page while offering navigational information such as scroll rates to show how far people move down the page before bouncing off.
Essentially an inversion of heat mapping, focus mapping provides digestible visual insights on the main fixation points on a specific page.
With focus maps, the page is blanked out except for the spots that receive the most attention or fixation.
A visual technique to complement additional eye tracking tests and consumer research strategies, with focus mapping you will get a panoramic view of which elements are working as well as the content you need to improve to encourage focus and engagement.
Gaze path plots
As sensory-based technology, eye tracking can provide a wealth of valuable insight in a single browsing session.
By adding metrics related to time as part of your eye tracking strategy, you can follow the path a user takes on a webpage and the time they spent on each element.
As you can see from the ‘Where’s Wally’ video, gaze plots make an effective eye tracking technique as they offer a dynamic interpretation of how users interact with your site or mobile app.
If you follow these paths, it’s possible to get a real-time insight into the eyes of your audience. This wealth of visual eye tracking data will empower you to drill down into specific areas of a web or app page, making design or content tweaks to optimize the overall user experience.
Eye tracking metrics
In addition to diversifying your approach to eye tracking and working with a mix tool or platforms, focusing on the right metrics will improve your chances of success exponentially.
Here are the main eye tracking metrics you should work with during your user research tests and studies:
Areas of Interest (AOIs): Before running an eye tracking test or study, you should determine your AOIs. Mapping out your main areas of interest on a particular web or mobile page will give your test definitive direction while ensuring you only collect or concentrate data to provide answers to the right questions.
Dwell time: This metric is focused on the actual amount of time a user or test subject spends interacting with a predetermined AOI. You can, for instance run A/B testing to compare two versions of a webpage to see which one returns the healthiest AOI dwell times and thus, offer the best return on investment (ROI).
Fixation count: Like dwell time, fixation count can offer interaction data based on your AOIs. But, rather than quantifying time, fixation count records up how many fixations your AOIs receive during a test or time period. As such, you can compare fixation and dwell time data to identify a correlation while painting a panoramic picture that will make your optimization efforts more successful.
Ratio: In an eye tracking context, ratio will tell you how many users or test subjects have guided their gaze to a particular AOI. Tracking ratio will give you an insight into whether you need to alter a specific piece of content or design element to capture the attention of more users and streamline your site navigation.
Revisits: Based on your AOIs, revisits determine how many times a user returns their gaze to a specific point during a test or browsing session. This metric is beneficial for understanding whether your design and layout make it easy for people to navigate a page or not. If you find that a particular AOI is showing a high rate of revisit rates, it might be that your content is a little confusing or that there is a design element causing distraction.
Getting started with eye tracking: best practices
Now that you understand how eye tracking works and you’re acquainted with essential techniques, we’re going to look at some top tips to help you get the most from your eye tracking test efforts:
Once you’ve established the aims and criteria of your test, allow your users to complete the process uninterrupted. Asking questions during the test itself is likely to distract your test subjects, skewing your results in the process.
Make sure that your participants remain within the monitoring range from start to finish. If a subject falls out of position, halt the test to ensure they are back in the monitoring range. Doing so will ensure the best quality data.
For qualitative testing and manual eye tracking, around five test subjects will typically offer the level of insight you need for the job. For heat maps and broader eye tracking studies, it’s recommended to use at least 39 test subjects for well-rounded and actionable results.
For mobile optimization tests, you should focus on the performance and value of your functional icons across devices; the precision of your error messaging; and the consistency as well as responsibility of your mobile design and layout.
To maximize your eye tracking efforts, working with customer experience optimization experts will help you run successful eye tracking-type tests that will return data that is aligned with your specific strategies and goals.
Final thoughts
“You can’t solve a problem on the same level that it was created. You have to rise above it to the next level.”
—Albert Einstein
Niche or sector aside, without knowing your audience and understanding how they interact with your business, you are merely shooting in the dark.
In an ever-evolving digital age, a data-driven approach to consumer research and UX optimization will help you meet your users’ needs head on while empowering you to adapt to constant change.
Eye tracking may not be the answer to all of your consumer-centric needs but this most innovative of sensor technology could easily play an important role in your ongoing marketing and development strategy.
There is no one-size-fits-all way to approach eye tracking—success will depend on your goals and needs as a business. But, by embracing the methods and techniques we’ve discussed, you will open yourself up to a treasure trove of data-driven insights that will accelerate the growth of your business.
We hope our guide to eye tracking has helped you on your way and for more business-boosting pearls of wisdom!
A Call to Action, also known as a CTA, refers, in marketing, to any item that will, using imperative wording, encourage an immediate action or response from the user.
Calls to Actionare essential in marketing campaigns. They’re a way to lead customers to a specific action. They generally come as a button, but they also exist in many other forms. In this guide, we’ll tell you everything you need to know about CTAs, how they work and how to use them on your website.
What is a Call to Action?
Behind this mysterious name hides a very simple marketing concept you’ve all seen before. Calls to Action (CTAs) refer to any device conceived to persuade users to do a specific action.
E-commerce companies usually use them in the form of buttons to encourage buyers to add an item to their shopping cart or to complete a transaction. It is a key element to integrate playful interaction with your users and an effective means to increase conversions.
The goal of a CTA is to use a word or a sentence (most of the time containing action verbs) to push your users toward a specific action like “click here”, “subscribe”, “check out” and many others. These action words can also be used with a “now” creating a sense of urgency. Calls to Action were proven to be very effective and to optimize conversion rates.
CTAs can be used to push users down the purchase funnel, but they can also be used for any kind of action like registering, subscribing to a newsletter or adding to cart.
What makes an effective CTA? When creating a CTA for your website, every detail counts. Here are some aspects you need to pay attention to for your CTA button:
Visual aspect
Wording
Action word
Placement
Form
Color
Size
Do not underestimate the impact your CTA button can have on your conversion rate. A well-written Call to Action needs to be adapted to your audience, their age, their gender or their nationality. Remember that a CTA isn’t just a command or an invitation for your users, it is part of the full purchasing process. That’s why it needs to be discreetly integrated but obvious enough to be noticed by users.
Why do you need a clear Call to Action in your CRO strategy?
Always remember the importance of CTAs in your purchase funnel. Your customer’s path to completing a transaction is paved with CTA buttons. They are a key element in your CRO (Conversion Rate Optimization) strategy, they need to be quite obvious since they are meant to lead users to the action you want them to complete.
A good and clear Call to Action comes with a nice visual, adapted to your target audience, with a clear and straightforward message. That way, and only that way, you will get the best out of your CTA and note effective results in your conversion rate AND your CTR (Click Through Rates).
An efficient CTA is nothing but a perfect compromise between your e-commerce site’s goal, which is to increase sales or to sell a specific product, and users’ needs, i.e. a smooth navigation experience while purchasing a product. That’s why your CTA needs to be neat and easy to find, and user experience always needs to be taken into account.
Best Call to Action examples of 2020
CTAs come in many forms. To help you with your CTA A/B testing, we listed the 5 best forms of CTAs in 2020:
1. Direct Calls to Action
Let’s say your goal is to push your customers down the purchase funnel. Then, you might opt for a direct CTA, such as:
Shop now
Buy now
Add to shopping cart
An Amazon product page containing an add to shopping cart and buying button CTA (Image Source).
2. “Get for Free” Calls to Action
With these CTAs, you can highlight an opportunity for the users. Generally, these come with a “subscribe” box. You can collect your users’ email addresses in exchange for a sample of a document or a free trial. These CTAs usually appear as:
Download for free
Free 30 day trial
Start free trial
Tidal homepage enticing users to register with a free trial (Image Source).
3. Basic Calls to Action examples
CTAs can be a mere invitation. For instance, on a social media’s ad or as a shortcut to a long text. This kind of CTA is often used in blog posts or Facebooks Ads. Awakening the user’s curiosity, it is supposed to make users want to go further and learn more about a topic with messages like:
Check it out
Start here
Find out more
Learn more
Philips USA homepage with multiple CTAs (Image Source).
4. Registering Calls to Action
This kind of CTA is often found on social media or e-commerce sites. The goal is simple: encourage your visitors to create an account and register with messages such as:
All of the above have one thing in common: they allow you to connect with your users by collecting their emails, which will be useful for your email marketing campaigns. According to what you are proposing, users can fill in their email to get something like a discount, a coupon or a free PDF. For these, you can use wordings such as:
A/B testing is the most accurate technique for CRO. This digital marketing strategy consists of making changes on your website and observing the impact of this change on a segment of users.
This is the best method to improve your conversion rate, because you can try out any feature and choose the one whose results are best. It is more reliable since users are the ones determining which feature works best. With this method, you can test your idea with your target users or potential customers.
A/B testing your CTA buttons is the best way to improve your website’s UX and your conversion rate at the same time. At AB Tasty, we offer a super quick and easy way to run these kinds of tests – our drag and drop visual editor.
How to run an A/B test for your CTA?
Here are the different steps you need to follow to A/B test your Call to Action button:
Define your test’s goal and the KPIs you want to improve
Changing your button’s feature must serve a goal. It wouldn’t make sense to change your CTA’s color, to run your test and wait for random results. Your goal can be to increase the number of users that subscribe to your newsletter, for instance.
Define the original and alternative version (A and B version)
Choose the CTA you want to run your A/B test on, let’s say the red “subscribe to newsletter” button. This red button will be your A version.
What do you want to change about it? Maybe users will be more likely to click on this button if it were blue? Then, change your button’s color. (If you’re using AB Tasty, this takes two seconds with our drag and drop editor). The blue version of the CTA button will be your B version.
Run your A/B test
Worried about exposing 50% of your website visitors to this new change? If you have enough traffic, you can run your test on a smaller percentage of your entire website audience to mitigate the risk of any potential lost conversions.
Collect data and check your analytics
Usually, A/B tests take several weeks before you get reliable results. During that period, observe your A/B testing tool reporting. Did your conversion rate increase? Did it decrease?
Hard code changes (or not)
Figures don’t lie. Based on the test’s results, change what needs to be changed and keep what needs to be kept. Step by step you will find the best combination of features for the perfect CTA.
Conclusion
CTAs are essential in your conversion rate optimization strategy, but they can’t appear randomly or look like any old button. They need to be wisely thought out, otherwise they might not have the expected effect.
Thanks to A/B testing, you will be able to find the right features in order to get the best performance out of them. Remember that visual aspects (like form, color, or size), wording, the chosen action word, and placementmatter in your CTA’s performance.
So, what’s the takeaway? An effective Call to Action can boost your website KPIs, and you can A/B test any of your website’s pages – the sky’s the limit.
Ronny Kohavi (ex-Amazon, Airbnb, Microsoft) 
Chad Sanderson (Convoy)
Jonny Longden (Journey Further)
