“Failure” can feel like a dirty word in the world of experimentation. Your team spends time thinking through a hypothesis, crafting a test, and finally when it rolls out … it falls flat. While it can feel daunting to see negative results from your a/b tests, you have gained valuable insights that can help you make data-driven, strategic decisions for your next experiment. Your “failure” becomes a learning opportunity.
Embracing the risk of negative results is a necessary part of building a culture of experimentation. On the first episode of the 1,000 Experiments Club podcast, Ronny Kohavi (formerly of Airbnb, Microsoft, and Amazon) shared that experimentation is a time where you will “fail fast and pivot fast.” As he learned while leading experimentation teams for the largest tech companies, your idea might fail. But it is your next idea that could be the solution you were seeking.
“There’s a lot to learn from these experiments: Did it work very well for the segment you were going after, but it affected another one? Learning what happened and why will lead to developing future strategies and being successful,” shares Ronny.
In order to build a culture of experimentation, you need to embrace the failures that come with it. By viewing negative results as learning opportunities, you build trust within your team and encourage them to seek creative solutions rather than playing it safe. Here are just a few benefits to embracing “failures” in experimentation:
Encourage curiosity: With AB Tasty, you can test your ideas quickly and easily. You can bypass lengthy implementations and complex coding. Every idea can be explored immediately and if it fails, you can get the next idea up and running without losing speed, saving you precious time and money.
Eliminate your risks without a blind rollout: Testing out changes on a few pages or with a small audience size can help you gather insights in a more controlled environment before planning larger-scale rollouts.
Strengthen hypotheses: It’s easy to fall prey to confirmation bias when you are afraid of failure. Testing out a hypothesis with a/b testing and receiving negative results confirms that your control is still your strongest performer, and you’ll have data to support the fact that you are moving in the right direction.
Validate existing positive results: Experimentation helps determine what small changes can drive a big impact with your audience. Comparing negative a/b test results against positive results for similar experiments can help to determine if the positive metrics stand the test of time, or if an isolated event caused skewed results.
In a controlled, time-limited environment, your experiment can help you learn very quickly if the changes you have made are going to support your hypothesis. Whether your experiment produces positive or negative results, you will gain valuable insights about your audience. As long as you are leveraging those new insights to build new hypotheses, your negative results will never be a “failure.” Instead, the biggest risk would be allowing a status quo continuing to go unchecked.
“Your ability to iterate quickly is a differentiation,” shares Ronny. “If you’re able to run more experiments and a certain percentage are pass/fail, this ability to try ideas is key.”
Below are some examples of real-world a/b tests and the crucial learnings that came from each experiment:
Lesson learned: Removing “Add to Basket” CTAs decreased conversion
In this experiment, our beauty/cosmetics client tested removing the “Add to Basket” CTA from their product pages. The idea behind this was to test if users would be more interested in clicking through to the individual pages, leading to a higher conversion rate. The results? While there was a 0.4% increase in visitors clicking “Add to Basket,” conversions were down by 2%. The team took this as proof that the original version of the website was working properly, and they were able to reinvest their time and effort into other projects.
Lesson learned: Busy form fields led to decreased leads
A banking client wanted to test if adjusting their standard request form would drive passage to step 2 and ultimately increase the number of leads from form submissions. The test focused on the mandatory business identification number field, adding a pop-up explaining what the field meant in the hopes of reducing form abandonment. The results? They saw a 22% decrease in leads as well as a 16% decrease in the number of visitors continuing to step 2 of the form. The team’s takeaways from this experiment were that in trying to be helpful and explain this field, their visitors were overwhelmed with information. The original version was the winner of this experiment, and the team saved themselves a huge potential loss from hardcoding the new form field.
The team at this beauty company designed an experiment to test whether displaying a message about product availability on the basket page would lead to an increase in conversions by appealing to the customer’s sense of FOMO. Instead, the results proved inconclusive. The conversion rate increased by 1%, but access to checkout and the average order value decreased by 2% and 0.7% respectively. The team determined that without the desired increase in their key metrics, it was not worth investing the time and resources needed to implement the change on the website. Instead, they leveraged their experiment data to help drive their website optimization roadmap and identify other areas of improvement.
Despite negative results, the teams in all three experiments leveraged these valuable insights to quickly readjust their strategy and identify other places for improvement on their website. By reframing the negative results of failed a/b tests into learning opportunities, the customer experience became their driver for innovation instead of untested ideas from an echo chamber.
Jeff Copetas, VP of E-Commerce & Digital at Avid, stresses the importance of figuring out who you are listening to when building out an experimentation roadmap. “[At Avid] we had to move from a mindset of ‘I think …’ to ‘let’s test and learn,’ by taking the albatross of opinions out of our decision-making process,” Jeff recalls. “You can make a pretty website, but if it doesn’t perform well and you’re not learning what drives conversion, then all you have is a pretty website that doesn’t perform.”
Through testing you are collecting data on how customers are experiencing your website, which will always prove to be more valuable than never testing the status quo. Are you seeking inspiration for your next experiment? We’ve gathered insights from 50 trusted brands around the world to understand the tests they’ve tried, the lessons they’ve learned, and the successes they’ve had.
In an increasingly cutthroat digital age, standing out in your niche while meeting the exact needs of your consumers is essential to business growth and longevity.
By getting under the skin of your customers, you can tailor your messaging, applications, and touchpoints to meet their exact needs —that’s where eye tracking enters the mix.
Consider this for a moment:
You’re running a usability test on a product landing page for a new range of gym shoes. Your test subject, Nancy, browses the page and chooses a shiny new pair of gym shoes with ease. But, on the next page, there is a snag. She hesitates and eventually abandons her cart because the journey was confusing.
You take notes based on Nancy’s feedback and think about how you can improve your checkout journey. But, if you could view her movements— or see what she sees—you would have the power to make informed improvements that will ultimately increase conversions and drive more sales.
With eye tracking, you can. But while this widely-used sensor-based technology offers a deep glimpse into user browsing behavior, some industry experts believe that eye tracking is an unnecessary expense.
Like many platforms and digital innovations, with the right approach, eye tracking will give you the tools to offer your customers a seamless level of user experience (UX)—the kind that will increase loyalty while helping you boost your bottom line.
Here we explore the dynamics of eye tracking and explain why it could make an excellent investment for your business.
So, what is eye tracking?
Eye tracking is a type of sensor technology that gives a computer or mobile device the tools to understand and trace where a person is looking.
An eye tracker can detect the presence, attention and focus of a user while engaging with a specific app, touchpoint or website.
From a marketing perspective, eye tracking dates back to the 1980s, where it was used to test and measure the value of ads in print papers or magazines.
An effective alternative to lie detection-style techniques such as voice stress analysis and galvanic skin response (neither of which offer truly reliable results or data), eye tracking gave the advertisers of the day essential insights into which elements of a page people read as well as how long they spend engaging with specific pieces of content.
The popularity of the eye tracker rose over the years and the rapid evolution of digital technology paved the way for a wealth of innovative developments.
Now, eye tracking technology is able to offer deep-dive insights into user behavior and dynamic page as well as app design as well as offering intuitive tools that enhance the user journey for disabled people.
In the modern age, one of the most prominent features of eye tracking is a little something called Facial Expression Analysis (FEA).
Based on ‘points of fixation’—times during the user journey when someone stops and focuses long enough to process the content before them (commonly known as a ‘saccade’)—FEA technology helps marketers gauge the effectiveness of their page design and messaging.
But, how does this apply to business and why is it so useful? Let’s find out.
As a marketer or business owner, the more you understand your target audience, the more chance you have of creating a fluent and engaging customer journey across platforms.
As eye tracking provides a visual map of how your users engage with your website, landing pages, and mobile applications, you can identify strengths and weaknesses related to user experience (UX) and content placement.
An essential part of the consumer research process, eye tracking is a powerful medium as it taps into the fact that 95% of human decision-making (particularly online) is carried out sub-consciously.
By using eye tracker tools to trace navigational patterns, you can adopt your customers’ vision, uncovering information that will help you to make improvements that boost engagement, improve your customer experience (CX) offerings, and ultimately, accelerate the growth of your business.
From heat mapping to task-based usability tools, there are a wealth of eye tracking innovations available to businesses in today’s digital world.
Invest in the right eye tracking tool for your business and you will:
Understand what your target audience is looking at and for how long
Identify redundant or disruptive visuals or design elements
Document how users scan and interact with your web pages or apps
Gain a practical understanding of what works and what doesn’t
Prove the value of certain marketing strategies, techniques or campaigns
Continually improve and evolve your efforts in a landscape that is ever-changing
Eye tracking is an effective means of seeing through the lens of your customers. But, as powerful as it is, eye trackers alone are unlikely to give you a complete insight into the content that really sticks in the users’ mind.
To gain additional context on how to use your data to improve usability and drive engagement, eye tracking should be a pivotal part of your consumer research strategy rather than a sole means of information.
That said, if you use it the right way, eye tracking can help you understand your customers in ways that can give you an all-important edge on the competition.
How eye tracking can help you understand your customers
Using eye tracking to understand your customers on a deeper level boils down to adopting a cohesive mix of the right tools and techniques.
Eye tracking tools and software provide a visual representation of your users’ focus points—returning data based on:
Fixation points or saccades: information that can tell you how engaging or eye-catching particular elements or pieces of content on a webpage are to your customers.
Navigational patterns: by understanding common navigational patterns, you can see how people scan or interact with your page. This level of knowledge will give you the data you need to optimize your content and design for increased engagement and conversions.
Problematic elements As mentioned, an eye tracking test will return invaluable data based on any images, graphics, calls to action (CTA) or command buttons, informational content or design elements that hinder the user experience and prevent customers from either getting what they need from your page or carrying out a desired action (clicking through to a specific product page or signing up to an email newsletter, etc).
Automotive repair and wreckage company, Truckers Assist, conducted eye tracking tests to track the performance of its homepage.
This test showed that while the ‘NO FEES’ graphic (the red point on the image) was gaining a lot of attention, it wasn’t clickable. As a result, many users were focusing their attention in the wrong place, steering them away from more valuable information as a result.
To fix this glaring issue, Trucker Assist improved its homepage design, removing the ‘NO FEES’ banner and placing focus on its contact information and service search bar.
Conducting successful eye tracking testing takes consistency as well as a clear cut goal. Do you want to improve the user journey of your new mobile app? Are you looking to drive more revenue through a specific product page? Perhaps you’re trying to understand if your general messaging and branding is performing the way it should?
There are many actionable insights you can gain from eye tracking—and outlining your specific goals will give your tests or studies direction.
This hand-picked video offers practical advice and information to help you get started with eye tracking:
How eye tracking benefits UX optimization
88%of consumers are less likely to return to a website after a poor user experience. Today’s consumers expect a seamless level of UX from brands and businesses—anything less and you could see customer loyalty as well as sales drop through the floor.
Eye tracking and UX go hand in hand. Through eye tracking, you will gain access to objective and unbiased insights that will show you where improvements are necessary.
With eye trackers, you can drill down into a specific UI element (is it facilitating the right interactions or are your consumers missing it altogether?) to test whether it fits into the user journey while getting to the very root of any distracting, problematic or misleading page elements.
This perfect storm of on-page information will empower you to make very specific improvements to any app, web or landing page—enhancing its usability and performance significantly.
How eye tracking works in a nutshell
As a concept, a significant part of eye tracking is based in Fitts’ Law. Essentially, every visual object or element carries a certain amount of ‘weight’ and this determines the amount of attention as well as clicks it ultimately earns.
Concerning eye tracking and UX, Fitts’s Law is important because it can help you predict the amount of time taken to move the eyes or cursor to a specific target.
Armed with this information, you can establish a visual hierarchy and optimize your webpages or applications to ensure consumers can connect with the right functions or information at the right times within their journey.
To get your eye tracking tests off to the best possible start, giving you users clearcut instructions while ensuring good lighting and consistent positioning is essential. Doing so will give you reliable data, as detailed in this infographic from IMOTIONS:
Essential eye tracking methods & techniques
There are thousands of eye tracking tools and countless ways of approaching this most powerful approach to user testing.
To guide you along the right path, here we’re going to explore the most essential eye tracking methods and techniques.
Heat maps
A branch of eye tracking, a heat map is a dynamic tool that offers a definitive visual representation of where users focus their attention and how they navigate your website based on their on-page interactions.
Heat mapping platforms provide color-coded data to give an indication of the areas of a website or mobile page users are interacting with the most.
As you can see from the image above, the red spots show the areas where users focus their attention most while the lighter colors are the areas with the lowest engagements.
Heat mapping technology also serves concrete data based on how much particular buttons or links are clicked by users on a page while offering navigational information such as scroll rates to show how far people move down the page before bouncing off.
Essentially an inversion of heat mapping, focus mapping provides digestible visual insights on the main fixation points on a specific page.
With focus maps, the page is blanked out except for the spots that receive the most attention or fixation.
A visual technique to complement additional eye tracking tests and consumer research strategies, with focus mapping you will get a panoramic view of which elements are working as well as the content you need to improve to encourage focus and engagement.
Gaze path plots
As sensory-based technology, eye tracking can provide a wealth of valuable insight in a single browsing session.
By adding metrics related to time as part of your eye tracking strategy, you can follow the path a user takes on a webpage and the time they spent on each element.
As you can see from the ‘Where’s Wally’ video, gaze plots make an effective eye tracking technique as they offer a dynamic interpretation of how users interact with your site or mobile app.
If you follow these paths, it’s possible to get a real-time insight into the eyes of your audience. This wealth of visual eye tracking data will empower you to drill down into specific areas of a web or app page, making design or content tweaks to optimize the overall user experience.
Eye tracking metrics
In addition to diversifying your approach to eye tracking and working with a mix tool or platforms, focusing on the right metrics will improve your chances of success exponentially.
Here are the main eye tracking metrics you should work with during your user research tests and studies:
Areas of Interest (AOIs): Before running an eye tracking test or study, you should determine your AOIs. Mapping out your main areas of interest on a particular web or mobile page will give your test definitive direction while ensuring you only collect or concentrate data to provide answers to the right questions.
Dwell time: This metric is focused on the actual amount of time a user or test subject spends interacting with a predetermined AOI. You can, for instance run A/B testing to compare two versions of a webpage to see which one returns the healthiest AOI dwell times and thus, offer the best return on investment (ROI).
Fixation count: Like dwell time, fixation count can offer interaction data based on your AOIs. But, rather than quantifying time, fixation count records up how many fixations your AOIs receive during a test or time period. As such, you can compare fixation and dwell time data to identify a correlation while painting a panoramic picture that will make your optimization efforts more successful.
Ratio: In an eye tracking context, ratio will tell you how many users or test subjects have guided their gaze to a particular AOI. Tracking ratio will give you an insight into whether you need to alter a specific piece of content or design element to capture the attention of more users and streamline your site navigation.
Revisits: Based on your AOIs, revisits determine how many times a user returns their gaze to a specific point during a test or browsing session. This metric is beneficial for understanding whether your design and layout make it easy for people to navigate a page or not. If you find that a particular AOI is showing a high rate of revisit rates, it might be that your content is a little confusing or that there is a design element causing distraction.
Getting started with eye tracking: best practices
Now that you understand how eye tracking works and you’re acquainted with essential techniques, we’re going to look at some top tips to help you get the most from your eye tracking test efforts:
Once you’ve established the aims and criteria of your test, allow your users to complete the process uninterrupted. Asking questions during the test itself is likely to distract your test subjects, skewing your results in the process.
Make sure that your participants remain within the monitoring range from start to finish. If a subject falls out of position, halt the test to ensure they are back in the monitoring range. Doing so will ensure the best quality data.
For qualitative testing and manual eye tracking, around five test subjects will typically offer the level of insight you need for the job. For heat maps and broader eye tracking studies, it’s recommended to use at least 39 test subjects for well-rounded and actionable results.
For mobile optimization tests, you should focus on the performance and value of your functional icons across devices; the precision of your error messaging; and the consistency as well as responsibility of your mobile design and layout.
To maximize your eye tracking efforts, working with customer experience optimization experts will help you run successful eye tracking-type tests that will return data that is aligned with your specific strategies and goals.
Final thoughts
“You can’t solve a problem on the same level that it was created. You have to rise above it to the next level.”
—Albert Einstein
Niche or sector aside, without knowing your audience and understanding how they interact with your business, you are merely shooting in the dark.
In an ever-evolving digital age, a data-driven approach to consumer research and UX optimization will help you meet your users’ needs head on while empowering you to adapt to constant change.
Eye tracking may not be the answer to all of your consumer-centric needs but this most innovative of sensor technology could easily play an important role in your ongoing marketing and development strategy.
There is no one-size-fits-all way to approach eye tracking—success will depend on your goals and needs as a business. But, by embracing the methods and techniques we’ve discussed, you will open yourself up to a treasure trove of data-driven insights that will accelerate the growth of your business.
We hope our guide to eye tracking has helped you on your way and for more business-boosting pearls of wisdom!
The sign of an excellent product or service is one that’s designed to suit the customer – whether that’s addressing a problem they may have or simply providing them with an experience that puts them at the center of it. Whether you’re running a B2B or B2C business, gaining customer feedback on their experience is essential for growing and developing your business, as well as improving on areas they found issues with. Customer satisfaction is imperative for the ongoing success of any business but it can sometimes prove to be rather tricky to get customers to give feedback on their experience in the first place.
Unlike NPS surveys where the purpose is to keep it short and sweet, the user satisfaction survey allows businesses to ask more in-depth questions and can enable customers to give more details about their experiences. But how do you write a survey that engages and interacts with customers enough to gain responses from them?
We explore some great tips to writing good user satisfaction surveys that will not only get you the responses you’re looking for, but help improve your business based on previous customer experiences.
Open With The Right Questions
In a previous article on NPS surveys, I touched on starting with the ‘right’ questions and the same applies for starting with a good user satisfaction survey – start by asking them directly about the product or service they have just used. Opening a satisfaction survey with the right type of question will enable customers to answer directly, avoiding any vague responses or possible misinterpretation. After all, you’re looking for constructive feedback about your business, so make sure your questions allow them to do this. Here are some great opening questions to try:
How often do you use the product or service?
What do you like best about the product or service?
What area would you improve on?
Encourage customers to give you more detailed responses with open text replies. By giving them the freedom and space to explain or describe their experience in detail, you will gain much more insight and obtain more information than simply asking them to tick a box or answer a simple yes or no.
Use Scales or Star Ratings
Having a scale of 1-10 or 1-5 will allow customers to measure their satisfaction by attributing a number to their experience. It’s up to you to decide which way the scale goes – with 1 being best or worst. Having a scale can be a great way to quantify satisfaction from customers when using a particular service, for example, but it can also mean customers don’t feel able to share their experiences as they give a number instead. A great way to tackle this is to elaborate after your scale, with a leading question like:
‘Why did you rate your experience with this score?’
The leading question after your scale will empower customers who do want to give a bit more detail about why they scored your business they way they did. There will be a number of customers who don’t feel compelled to elaborate on why they have scored your business with that particular number, so make sure to use their score to help understand what kind of experience they had and perhaps how you can improve for next time.
Remember, the whole purpose is to improve your business from their feedback so if you use a scale, make sure you are clear on what the numbers mean and apply metrics that will look to improve your product or service.
Include Yes Or No Questions
A great user satisfaction survey should have a mix of quick answers and ‘open-ended’ options to allow customers to give more details should they want to. Some customers are going to be immediately put off when faced with lots of open-ended questions, as they won’t want to take the time to give you the appropriate feedback, so make sure to mix it up. Include some simple ‘yes ‘ or ‘no’ style questions too to encourage those customers who are time-poor, to answer your survey quickly. Try questions like:
Would you use our services again?
Would you recommend us to friends and family?
Did our services meet your expectations? – Ask ‘if not, could you give us a bit more detail’ to allow a customer to describe their experience
Did you find what you were looking for? – Try offering different responses if they click yes or no here if no – ask them why
Yes or no answers may not appear to give a company enough feedback, but when followed up with a further leading question and open answer, it empowers customers to divulge more about their experience, allowing the brand to learn and look to improve on the particular experience.
Start to Explore Demographics
This isn’t compulsory for your customer satisfaction guide but it does really help businesses develop if they have a better profile and understanding of their demographics, as you can really start to segment your audience. If you haven’t already, try to gain further information from existing customers about their personal situation: how old they are, what their employment status is and even where they are in the world. Obtaining this kind of information can really help businesses improve their personalization approach whilst creating a more accurate depiction of the types of customers the brand appeals to. Try asking the following:
How old are you? – We recommend giving a range of ages, e.g.: 35-50, 51-65
Where do you live? – Again, if being town-specific will help, use specific locations. Otherwise, per country is fine
What is your marital status? – Be inclusive and try not to unintentionally alienate anyone
What is your employment status? – Try to angle it with ‘self-employed’, ‘Contractor’, etc
Try Multiple Choice Questions
If you’re looking to create customer profiles from their survey answers, try incorporating some multiple-choice questions, too. Giving customers options to choose from often encourages them to complete your survey, as it means they will spend less time overall giving their feedback, but also means you can obtain further details about them as an individual as well as their experience with your brand. Try incorporating the following questions in your next user satisfaction survey:
Which best describes the reason for your visit today – ‘Leisure’, ‘Business’, ‘just browsing’ etc
What services/products have you bought from us in the past? Are you looking for similar products/ services?
What was your main reason for visiting the website/ store today – To find out more information? to purchase a specific item? to speak to us directly?
These multiple choice answers give enough details about the customers’ experience for business to learn from and will appear to ‘save time’ as they offer an answer instead of openly asking customers to respond – a win, win situation for both business and customer!
Use It To Test New Experiences
User satisfaction surveys aren’t exclusive to improving customer experience, they can also be used when looking to launch new products or test a new digital experience, such as a new website or online journey. The best way to get good, productive feedback on prototypes or new designs is to let customers use them and give their own feedback on how it went!
For instance, if you have developed a new website or want to test how well received a new product is, give users the option to use it and create a survey around their experience. Twitter is a great example of a platform that looks for active user feedback each time it rolls out an update, by offering certain users access to their ‘new look’ platform. The survey allows for tweaks to be made, if required, and saves both time and money addressing any issues further down the line once the product is properly launched.
If you’re looking to try this approach try questions like:
Did you find what you were looking for today? A ‘Yes or No’ would be a great option here, with a follow-up if they respond ‘no’
How would you rate your experience using the new website? – Use a scale
What do you think of the new layout? – Perhaps offer multiple choice answers here to avoid vague answers
Ultimately, a good user satisfaction survey can use any or a combination of all of the above elements, as long as the overall intention of measuring satisfaction and experience is easily quantified, the way you ask customers doesn’t really matter. As long as the responses enable your business to learn and improve areas that customers highlight, the general structure of your survey is up to you!
Don’t forget – the journey of providing a great customer experience starts at the first point of contact with your business, whether that’s via social channels, adverts, email marketing or from your website, so next time you’re looking to improve things, start by looking at the experience a customer may get at these various touch points.
