Article

9min read

52 Email Subject Lines That Work From Fashion and Beauty Brands

You’ve all heard the saying before: You should never judge a book by its cover. Sounds ridiculous, right? Well, whether you like it or not, that’s exactly how people tend to determine an email’s worth by its subject line. 

To give you some perspective, 35% of email recipients open emails based on subject lines alone according to HubSpot, and 21% of subscribers mark boring ones as spam, even if they know they aren’t. 

In other words, customers are tough to impress. 

If you really want to get noticed and prove you are deserving of their time and attention, supercharge your email subject lines with emotion. Because in the end, no matter how much we rationalize, our buying decisions spring from our emotions

Read more: Creating Emotional Connections with Customers Using Data

Think of what a customer fears, dreams of, desires and wants in a product or solution. Use that as a premise of your subject lines to trigger an emotional response

Some fashion and beauty brands have already been doing consistently well in this area. It’s time you learn the ropes from them.

Fair warning: It’s not a good idea to rely on emotions alone. The purpose of retail brands, like yours, is to sell. So hit a variety of customer touchpoints, including sending personalized emails, because they are 26% more likely to be opened than those without. Even better, they deliver 6X higher transaction rates.

Anyway, let’s dive in…

Email subject lines that induce FOMO

What does FOMO mean? Fear of missing out.

No one likes missing out on time-sensitive deals, including you. Just think of the last time you jumped on an offer because you didn’t want to regret losing out on a good bargain. 

In another case, imagine a time when you felt some stress for not having bought something when there was still time, stock, and a good deal. 

The same emotional reaction happens to your customers. Whether it’s creating a top-notch holiday campaign, announcing a limited time or a limited stock sale, pick the right set of words like ‘last chance, ‘don’t miss…’, to create urgency. Seeing such offers immediately activates this fear of missing out and drives them to act right away. 

It’s exactly why this email from The North Face displays. The simple nudge is enough to get their recipient to grab this one– the last chance to save some money.

The North Face - using FOMO in email subject line

Examples of FOMO-provoking email subject lines:

Limited time offer:

  • Alice & Olivia: LAST.DAY.DROP.EVERYTHING.
  • Dropps: Get 15% off, but move quick.
  • UNIQLO: TICK TOCK! Free shipping ends tonight
  • Mango: LAST FEW HOURS to enjoy 30% off the entire collection!

Show it’s in demand: 

  • Nykaa: These will sell out in 3…2..⏳
  • Sorel: These stunners are selling out fast
  • Olay: Get it before it’s gone!?
  • Marc Jacobs: New to sale and going fast

Imply negative outcomes: 

  • Nykaa: You snooze, you lose.
  • Revolve: Say goodbye to 65% off ?
  • UNIQLO: Get it before it’s gone
  • Nike: Good stuff in your cart…checking out > missing out

More ideas:

  • You’ve still got time
  • Offer expires tonight
  • The stock’s running out
  • UH-oh. Your fav styles are (almost) gone
  • ATTN: This pack will DEFINITELY sell out
  • Saying goodbye is the toughest
  • Our top 10 at 10% off for the next 24 hours
  • LAST chance to save up to $55
  • <New product> is selling like hot cakes
  • You’ve ONE day left: FREE shipping +20% off
  • Tonight only: Your wish list items on sale
  • LIMITED-time offer on items on your wishlist

Email subject lines that drive curiosity

Can you guess what makes crime drama series binge-worthy? Why we click intriguing Upworthy and BuzzFeed posts? Or why we can’t stop reading novels? Blame our inherent need to close the loop between what we know and what we want to know. Otherwise known as – our curiosity!

This is also exactly why when people receive emails with partial information in the subject lines, they feel the urgent need to click ‘open.’ 

A perfect example is this email from Nykaa. This cliffhanger technique adds a touch of drama and mystery that is enough to tempt people to see what’s on the other side.

Nykaa - using curiosity in email subject line

Examples of using curiosity in subject lines: 

  • Huckberry: “Want” — everyone
  • Michael Kors: Give us an inch…
  • J.Crew: The shorts circuit
  • TwoThirds: Meet our unique pieces!
  • Revolve Tomorrow’s outfit forecast
  • Kate Spade: You’re getting sleepy…
  • Steve Madden: TORNADO warning
  • Anthropologie: Let us per-SUEDE you.
  • Birchbox: We’ve got some ideas for you.
  • Sephora: Because you need these.
  • Estée Lauder Online: Why’s everyone obsessed with retinol?
  • Jimmy Choo: Dreams are made up of these + complimentary global shipping

More ideas:

  • Not even in your wildest dreams would…
  • Got a sec? Open this email right away, or…
  • FIRST time ever on sale
  • New launches inside
  • Is THIS what you’ve been waiting for?
  • You DESERVE this
  • Don’t buy from us…
  • Before you regret not buying your favs
  • Saying goodbye is the toughest
  • Did we tell you that…
  • This is personal…

How to personalize and test subject lines to maximize your open rate

By crafting a personalized email with your customer’s name or a point of interest in the subject line, you’re ensuring a connection with your customer.

According to Forbes, 72% of customers ONLY interact with personalized messaging.

Personalization is an important way to build trust with your customer and maintain your relationship with them before, during, and after their purchases. If you’re not using personalization in your email campaigns, you’re likely missing out on a huge opportunity. 

Whether you’re looking to personalize your email content to capture customer attention or A/B test your subject lines to determine the best-performing phrase, choosing the right software will help you transform your ideas into reality.

AB Tasty is the complete experience optimization platform to help you create a richer digital experience for your customers — fast. From recommendations to smart search, this solution can help you achieve personalization with ease, experiment with a low-code implementation and revolutionize your brand and product experiences.

Email subject lines that spark happiness

Since you already occupy a sacred space in your customers’ inboxes, why not become a reason for their happiness?

A lot of things make people happy. So right from using humor, wordplay, solving people’s problems, appealing to their vanity, surprising them with freebies and special offers to reminding them there’s good in them. For example, if you can relate their shopping efforts to contributing to a social cause, that’s one idea of reminding them there’s good inside.  There are plenty of ideas that you can use to really catch their eyes and make them feel unique.

This email from Fortress of Inca, for example, plays the vanity card. Honest confession: I was at the receiving end and I had to open the email because my fabulous choice in shoes was being appreciated!

Fortress of Inca_ triggering happiness through email subject line

Examples of subject lines focusing on happiness:

  • Pura Vida: Save the Amazon Rainforest!
  • Michael Kors: FOOD IS LOVE: Help Us Watch Hunger Stop
  • Sephora: Your beauty issues solved
  • Nicole Miller: Button me up, Button me down.
  • Alice & Olivia: Hot date? We’ve got you.
  • J.Crew: We don’t do this (MAJOR) sale too often…
  • JustFab: Your FREE money is waiting. We put $30 in your account.
  • Pura Vida: Happy birthday to us (gifts for you!)
  • DSW: Find out how you can make a difference.
  • Patagonia: School lunch made easy
  • The North Face: 5 jackets that will have everyone saying…where’d you get them from!

More ideas: 

  • Our birthday treat = gifts for you!
  • Giving > Getting
  • It’s true…these will look dapper on you
  • What to wear for the New Year’s bash
  • Style secrets you can master in under 10 minutes
  • Your winter break packing list is HERE!
  • You’re one click away from shopping complete spring look
  • Impressed by items in your card: Buy ‘em before they’re gone
  • You earned what’s INSIDE this email

Email subject lines that create excitement

So far we’ve already established that humans are emotional creatures. Let’s now use excitement to get your sales meter ticking, because chances are that it results in impulse shopping.

Selling to customers is much easier when they’re excited and in touch with their emotions. In a state of excitement, your level of enthusiasm brings you to think and behave differently.

There are a number of ways to get the excitement level up. You can start by highlighting the exclusiveness of your offer, adding social proof (drop names and numbers), capitalizing action verbs to motivate action completion and even using emojis and exclamation marks.

In the email subject line below, team Pura Vida does two things — attract attention and excite the subscriber with the clever use of an emoji, the word ‘holy smokes,’ and a few ‘!

Pura Vida - triggering excitement in email subject line

Examples of Building Excitement in Subject Lines:

  • J.Crew: What Adam Scott thinks of our suits
  • Anthropologie: Just for you: the inside scoop
  • Pura Vida: Special offer (but only for our BFFs)
  • Olay: Over 50 million sold ❤️
  • Sorel: Meet our exclusive Frozen 2 boots✨
  • DSW: Reviewers have spoken: “BEST BOOTIE EVER!”
  • Steve Madden: Take our shoe stylists’ word for it
  • Meundies: Well, this is exciting!
  • Adidas: You’re in. Welcome to Adidas
  • Revolve: This collab was made for you
  • theBalm Cosmetics: Welcome to #THEBALMERS community!
  • Victoria’s Secret: 60% off bras & apparels, in stores only! GO, GO, GO!

More ideas: 

  • Beyonce-approved autumn wear you can’t say no to
  • Sweet launch discount (for members only!)
  • Howdie! Your faves are back in stock
  • Sshh…you’re the FIRST ONE to be seeing this
  • It’s between you and us ONLY
  • For your eyes only, because you wanted it
  • Baby, it’s baccccck!
  • It’s your birthday! Get 30% off, a FREE gift and free shipping.

Ready to Craft Email Subject Lines That Deliver?

Full disclaimer: It takes a lot of A/B testing and patience to come up with subject lines that stay within the 40-50 characters limit and stir the right emotions. Thankfully,  now you’re aware of which emotions to go after and how to invoke emotional decision-making.

So it’s time to get to work! But before you go, and for maximum impact, keep these equally important things in mind:

  • Match your brand personality with the tone of voice of your subject lines.
  • Segment your audience to send targeted/personalized emails.
  • A/B test your email send times.

Article

10min read

How to Define and Track your KPIs

Key Performance Indicators are one of the most important tools a business has at its disposal.

While the principle has existed for centuries, KPIs have taken off and reached their full potential thanks to the internet. This is particularly the case in measuring online engagements, such as sales or customer acquisition.

From free tools, such as Google Analytics, to highly complex software, there are many different types of KPIs measuring immensely diverse metrics.

What are key performance indicators (KPI)?

KPI, or Key Performance Indicators, are performance metrics that measure specific goals for businesses across all sectors.

Sometimes referred to as KSI (Key Success Indicators), when designed and implemented properly, they can define the direction of a business, provide essential feedback and help organize individuals, teams, projects or entire businesses to optimize performance.

A common theme throughout this article will be pointing out how KPIs differ vastly in how they are designed and what they measure. This is crucial to understanding their effectiveness and how they should be implemented and interpreted.

For example, high indicators might measure the overall performance of an E-commerce business (profits), while low indicators might measure sales of a specific product or output of a specific department.

In other words, KPIs can be helpful for departments, employees, managers, processes and even customer support teams.

Common things Key Performance Indicators might track are:

  • Revenue: average profits, total revenue, and new customers
  • Employment statistics: employee turnover, employee performance, and vacancies
  • Customer service: average call time, efficiency and customer satisfaction
  • Marketing: sales generation and overall effectiveness
  • Efficiency: overall efficiency, departmental processes and individual efficiency

How do you calculate a KPI?

Knowing how to measure a KPI is a matter of defining specific goals from the beginning. Focusing on the right KPI is vital and requires you to design it with a narrow scope.

A startup is likely to be more interested in tracking how many new customers are coming in rather than an established public company, which might be more focused on tracking share price and profit.

The most common tool for tracking KPIs is web analytics. Google Analytics is able to track a large quantity of data, from website performance to new subscribers, to sales. However, the issue with such hard data is that sometimes the metric that you want to specifically track is somewhat intangible or open to individual interpretation.

This may also become harder to measure when dealing with aspects of customer satisfaction. Typically, these types of metrics will require more than one key performance indicator, but it is important not to get carried away, as too much data can quickly become confusing.

One of the most important ways of tracking metrics through KPIs is related to presentation.

Google Analytics is a perfect example of how to present complex data in a clear way. Visualizations that can provide deep insights and deepen your understanding of the metrics will help make better sense of otherwise complex data.

Visual representations of data are a far better way of getting across important data instead of a dense presentation that provides no interactivity. The ease of viewing and digesting becomes particularly important when dealing with multiple KPIs.

It’s important to keep in mind that KPIs require constant evaluation to ensure they remain relevant and focused on the important parts of the business that need tracking.

How do you define a KPI?

Now let’s move on to two important questions:

  • What makes a KPI effective?
  • How do you define a KPI and cultivate metrics that provide insightful information?

An effective KPI depends on what you need to accomplish based on your current circumstance.

While it’s true that KPIs differ from sector to sector, they also vary from company to company. In fact, competitors with many of the same needs might differ wildly in their use of KPIs depending on philosophy and strategy.

The best place to start would be to understand the common use of KPIs, their philosophy, and their strategy within your own company and in your specific industry.

From here it is essential to define your goals. Before diving head first into the specifics of the KPI, you might want to look at it from a larger point of view.

For example, you can ask yourself where you might need to increase efficiency. This may take more time than originally accounted for, but the better the research, the more likely the KPI will harvest insightful results.

In the same manner, it’s also very important to set goals that are achievable.

KPIs are about focused data, not setting ambitious targets that can skew performance away from cohesive strategies.

One of the most effective ways of evaluating the effectiveness and appropriateness of a KPI is the SMART criteria. SMART stands for Specific, Measurable, Attainable, Relevant, Time-Bound:

  • How SPECIFIC is the goal?
  • Is it clearly MEASURABLE?
  • How ATTAINABLE is it?
  • Is it RELEVANT to the business?
  • What is the TIME Frame to achieve the objective?

SMART GoalsOnce all these criteria have been met, a KPI can be properly designed and implemented with confidence. However, it will need monitoring and adjustment as time goes on once the KPI has been fully integrated.

Different types of KPIs

As KPIs can be employed to measure deeply diverse metrics across a myriad of sectors and processes, types of KPIs also differ in how they are designed and implemented.

The core purpose might remain the same: provide concise data on various aspects of a business. However, the similarities often end there.

Three of the most common types of KPIs are designed for:

  1. Companies
  2. Teams
  3. Projects

Company KPIs

Although company KPIs might sound rather broad in their ambitions, they typically should be focused on specific areas of company performance. For customer metrics, these can be as diverse as acquisition, lifetime value of customers, retention, or customer loyalty.

On the employee side, KPIs are often used to measure certain business goals and performance – including measuring the strengths of specific employees.

For example, well-being is increasingly an important issue related to employee retention rates. Both well-being and employee retention rate can be gauged with the right tools to track their performance.

Team KPIs

KPIs for marketing require vastly different insights than KPIs for sales. In the same way, human resources, the finance team, legal, or any other department track and measure different KPIs. For this reason, key performance indicators for teams are one of the most unique.

A sales team will be likely to track customer capture, average deal size or revenue targets. On the other hand, the customer support team will likely focus on customer satisfaction by tracking results from various surveys or caller wait times. These two departments have some degree of crossover in their day-to-day work life, but they often have different KPIs they’re looking to measure that are more relevant for them.

In a different sense, the financial team will likely be tracking revenue, expenses, profit, and cash flows, which is a very different set of metrics compared to those of the sales team or customer support team.

Marketing teams, for their part, often require some of the most complex KPIs, with generated sales and brand awareness at the heart of their focus. This might include gathering quantitative data from an analytics source or gathering qualitative data.

KPIs for Human Resources (HR) teams have a less easily defined set of metrics to measure, such as the previously mentioned employee happiness and turnover. They may also measure how long it takes to fill a position and the number of responses to a given vacancy.

KPI Process

Project KPIs

The first thing that needs to be defined for a project is its goals, and this will define the KPI from the outset.

Measuring the value of a project is key to understanding how successful it is in a broad sense, which aspects are performing well, which elements are underperforming and how to cultivate fruitful and realistic goals.

KPIs for websites are the most numerous spanning across lead generation, sessions, bounce rates, e-commerce conversion rates, and sales all concisely presented on the platform.

It should be noted that employing too many KPIs for any aspect of a business can dilute its focus and confuse matters greatly. This is where careful preplanning and clearly defined goals can help.

KPIs vs OKRs

OKRs (Objectives and Key Results) are a newer and very commonly used variation of KPIs. In recent years, these have become increasingly popular, partly due to Google making the tracking process very simple and thorough.

There is a lot of overlap between the two, but the main difference is in their ambition.

OKRs are not defined as obtainable goals based on previous data, rather they are ambitious goals with clearly defined steps to achieve those goals. They should not be seen as unreachable targets but as motivational ones.

It would be wrong to compare the usefulness of KPIs and OKRs as they are designed with different outlooks on goals.

The main thing to keep in mind is that KPIs assess clear goals based on previous data while OKRs are better implemented for entirely new goals that require a broader vision.

Measuring and tracking KPIs

KPIs are only as successful as the template and interface they present. As previously mentioned, these are typically unique to the types of data they are measuring.

Website KPI examples tend to be well encapsulated by Google Analytics, with its drill-down data on broad, but related, topics such as conversion rates and returning visitors.

There are some types of metrics that you need to be aware of, such as vanity metrics.

Vanity metrics are one of the traps that businesses often fall into. This is where a superficial reading of positive data is seen as success alone, such as a growing number of visitors. With Google Analytics or any other measuring platform, it is essential to drill down further to find out what these numbers mean. For example, are these visitors the right type of visitors the business is looking for?

Google Analytics KPI Measurements

Social media KPI examples are typically well-designed and defined by in-depth on-site analytics. Twitter, for example, has a very effective template that offers the type of drill-down data that Google provides. Combined with other software, these can offer great insights regarding customer reach, acquisition and brand awareness.

Creating a strong KPI Strategy

Now that we have a clear picture of what KPIs mean and how to track them, you can start tracking your performance, aligning your team, and creating a KPI-focused culture.

Based on your organization’s objectives and the goals of your specific team, you’ll have to select the most important KPIs to track your success.

Once the KPIs are strategically chosen and implemented, keep note of their progress and downfalls and make adjustments as needed to get optimal results.

For a strong KPI strategy, it’s important to have your whole team aligned on organizational KPIs, team KPIs, and individual KPIs.

Having everyone on the same page will allow for transparency in tracking and performance reporting in the future. If no one understands the KPIs and how they are going to be tracked on any level, it will be difficult to build and implement a successful strategy.

Keep your team members clued in, educate them, assign them KPIs as needed, and use a user-friendly platform aligned with your goals to track your efforts.

As the market is constantly advancing, it’s important to be flexible and change your KPIs accordingly.

This is why it’s important to regularly meet and review your KPIs, the changing environment, and company policies. With any changes, it’s important to update your team accordingly to strengthen your strategy.

The importance of KPIs

There’s a reason why these performance indicators are known as the “keys.” KPIs are one of the most essential tools for performance measurements and have been around for a long time.

The implementation of KPIs is important to measure your success and follow your organization’s overall journey, but keep in mind that continuous tracking and measuring of KPIs is even more crucial to evaluate performance as you go along.

As they are customizable in many ways, key performance indicators are the best way to give you a quantifiable measure of a specific objective and give you and your team goals to aim for.

Article

5min read

1000 Experiments Club: A Conversation With Elissa Quinby of Quantum Metric

Elissa Quinby explains why a frugal mindset around experimentation can actually accelerate the process and increase resourcefulness.

Elissa Quinby lives and breathes retail, with eight years under her belt at Amazon working across multiple business units and functions on the marketing and product teams, as well as prior positions at Google and American Eagle Outfitters. 

Currently the Senior Director of Retail Marketing at Quantum Metric, an experience analytics company that helps brands to gain insights about their customers and make rapid, data-driven decisions, her expertise has been put to good use for the past year.

AB Tasty’s VP Marketing Marylin Montoya spoke with Elissa about ways to encourage loyalty from customers, methods for experimentation and how even the smallest piece of data can have a huge impact on tailoring the customer journey for a better overall experience.  

Here are some of the key takeaways from their conversation.

 

 

Start with ONE key piece of data from your customer and use it to build brand loyalty.

As marketers, we know the value of our current customer base, given the time, effort and cost of acquiring new customers. So it’s only logical to focus on improving the user experience in order to encourage repeat shoppers. 

During her time working at Amazon, Elissa adopted a mindset of frugality and learned how much of an impact can be made with only one piece of customer data. Today, she challenges retailers to ask themselves what data they already have that could revolutionize their customer experience.

With first-party data being the “secret sauce,” Elissa recommends starting small and offering value in return for their cooperation. Customers are increasingly hesitant to share their information with brands, so it’s important to offer an enticing incentive that will allow you to gather that one valuable piece of data that will improve the consumer experience. 

The hardest part of gathering that vital first-party data is encouraging customers to create an account. Once a customer has a profile, trust can be built over time and more data can be gathered, but always in exchange for value. For example, you can encourage customers to sign in to shop by offering personalized filtering or search results. This creates a more efficient and enjoyable online shopping experience for your customers as a reward for their loyalty.

 

“There’s literally nothing that should not be experimented on.”

Experimentation should be at the core of every marketing strategy. In a process of continual improvement, the possibilities for optimizing the customer journey are endless, however data is the only way to know for sure which modifications to pursue.  

With an emphasis on speed, the idea of experimentation is to test a new solution as quickly as possible, releasing any attachment to perfection, in order to start collecting customer feedback. 

Elissa explains that any new feature must be tested before it launches. Until customers offer feedback via their interactions, it remains a simple hypothesis to be proven. Not only does this save time on development, but you can gauge the user response to the experiment and make the necessary adjustments.

The experimentation process is precise, methodical and data-driven, to ensure the experiment is set up correctly for a reliable and insightful result – regardless of its success or failure. 

As the majority of tests do fail, it’s important to fail fast in order to learn as quickly as possible from the customers’ reaction. Elissa explains that running tests multiple times with slight adjustments can help to pinpoint the issue, which might be as simple as where in the customer journey a prompt is showing up. 

 

Experimentation tools can help brands optimize customer experience.

While manual methods for testing can yield results, an experimentation tool can supercharge your customer experience optimization. 

An experimentation tool not only saves time, but also ensures you are getting the most out of each test. It begins with data-driven ideation for the best hypotheses, and if your test fails to meet target metrics, a tool will allow you to pivot by ensuring that you have another hypothesis at the ready, also backed by data. 

Secondly, being able to pinpoint why an experiment failed, with comprehensive analysis, is key to improving your results without exhausting your resources. 

Finally, an experimentation tool can offer real-time data. If your experiment isn’t tracking well, you’ll know immediately and can shut it down. Conversely, if it’s a winner, you can start working with the product team to launch the new feature. It allows innovation cycles to be sped up, with decisions based on real-time data analysis of the user journey and browsing behavior. 

By optimizing the experimentation process with an intelligent analytics solution, you can improve efficiency and quickly hone in on features that are going to bring meaningful improvement to the customer experience and therefore drive results for the company. 

What else can you learn from our conversation with Elissa Quinby?

  • How to do more with less resources (both time and money)
  • How to stand out from competitors via a loyalty program
  • Why you should leverage digital during all phases of the customer journey
  • Why all customer insights play a vital role in improving business results

 

About Elissa Quinby

Elissa Quinby is an expert in retail insights, starting her career as an Assistant Buyer at American Eagle Outfitters followed by two years at Google as a Digital Marketing Strategist. She went on to spend eight years at Amazon across multiple business units and functions including marketing, program management and product.

Today, Elissa is the Senior Director of Retail Marketing at Quantum Metric, an experience analytics company that helps brands to gather customer insights which drive intelligent decision-making.

About 1,000 Experiments Club

The 1,000 Experiments Club is an AB Tasty-produced podcast hosted by Marylin Montoya, VP of Marketing at AB Tasty. Join Marylin and the Marketing team as they sit down with the most knowledgeable experts in the world of experimentation to uncover their insights on what it takes to build and run successful experimentation programs.

 

Article

6min read

Statistics: What are Type 1 and Type 2 Errors?

Statistical hypothesis testing implies that no test is ever 100% certain: that’s because we rely on probabilities to experiment.

When online marketers and scientists run hypothesis tests, they’re both looking for statistically relevant results. This means that the results of their tests have to be true within a range of probabilities (typically 95%).

Even though hypothesis tests are meant to be reliable, there are two types of errors that can still occur.

These errors are known as type 1 and type 2 errors (or type i and type ii errors).

Let’s dive in and understand what type 1 and type 2 errors are and the difference between the two.

Type 1 and Type 2 Errors explained

Understanding Type I Errors

Type 1 errors – often assimilated with false positives – happen in hypothesis testing when the null hypothesis is true but rejected. The null hypothesis is a general statement or default position that there is no relationship between two measured phenomena.

Simply put, type 1 errors are “false positives” – they happen when the tester validates a statistically significant difference even though there isn’t one.

Source

Type 1 errors have a probability of  “α” correlated to the level of confidence that you set. A test with a 95% confidence level means that there is a 5% chance of getting a type 1 error.

Consequences of a Type 1 Error

Why do type 1 errors occur? Type 1 errors can happen due to bad luck (the 5% chance has played against you) or because you didn’t respect the test duration and sample size initially set for your experiment.

Consequently, a type 1 error will bring in a false positive. This means that you will wrongfully assume that your hypothesis testing has worked even though it hasn’t.

In real-life situations, this could potentially mean losing possible sales due to a faulty assumption caused by the test.

Related: Sample Size Calculator for A/B Testing

A Real-Life Example of a Type 1 Error

Let’s say that you want to increase conversions on a banner displayed on your website. For that to work out, you’ve planned on adding an image to see if it increases conversions or not.

You start your A/B test by running a control version (A) against your variation (B) that contains the image. After 5 days, variation (B) outperforms the control version by a staggering 25% increase in conversions with an 85% level of confidence.

You stop the test and implement the image in your banner. However, after a month, you noticed that your month-to-month conversions have actually decreased.

That’s because you’ve encountered a type 1 error: your variation didn’t actually beat your control version in the long run.

Related: Frequentist vs Bayesian Methods in A/B Testing

Want to avoid these types of errors during your digital experiments?

AB Tasty is an a/b testing tool embedded with AI and automation that allows you to quickly set up experiments, track insights via our dashboard, and determine which route will increase your revenue.

Understanding Type II Errors

In the same way that type 1 errors are commonly referred to as “false positives”, type 2 errors are referred to as “false negatives”.

Type 2 errors happen when you inaccurately assume that no winner has been declared between a control version and a variation although there actually is a winner.

In more statistically accurate terms, type 2 errors happen when the null hypothesis is false and you subsequently fail to reject it.

If the probability of making a type 1 error is determined by “α”, the probability of a type 2 error is “β”. Beta depends on the power of the test (i.e the probability of not committing a type 2 error, which is equal to 1-β).

There are 3 parameters that can affect the power of a test:

  • Your sample size (n)
  • The significance level of your test (α)
  • The “true” value of your tested parameter (read more here)

Consequences of a Type 2 Error

Similarly to type 1 errors, type 2 errors can lead to false assumptions and poor decision-making that can result in lost sales or decreased profits.

Moreover, getting a false negative (without realizing it) can discredit your conversion optimization efforts even though you could have proven your hypothesis. This can be a discouraging turn of events that could happen to any CRO expert and/or digital marketer.

A Real-Life Example of a Type 2 Error

Let’s say that you run an e-commerce store that sells cosmetic products for consumers. In an attempt to increase conversions, you have the idea to implement social proof messaging on your product pages, like NYX Professional Makeup.

Social Proof Beispiel NYXYou launch an A/B test to see if the variation (B) could outperform your control version (A).

After a week, you do not notice any difference in conversions: both versions seem to convert at the same rate and you start questioning your assumption. Three days later, you stop the test and keep your product page as it is.

At this point, you assume that adding social proof messaging to your store didn’t have any effect on conversions.

Two weeks later, you hear that a competitor had added social proof messages at the same time and observed tangible gains in conversions. You decide to re-run the test for a month in order to get more statistically relevant results based on an increased level of confidence (say 95%).

After a month – surprise – you discover positive gains in conversions for the variation (B). Adding social proof messages under the purchase buttons on your product pages has indeed brought your company more sales than the control version.

That’s right – your first test encountered a type 2 error!

Why are Type I and Type II Errors Important?

Type one and type two errors are errors that we may encounter on a daily basis. It’s important to understand these errors and the impact that they can have on your daily life.

With type 1 errors you are making an incorrect assumption and can lose time and resources. Type 2 errors can result in a missed opportunity to change, enhance, and innovate a project.

To avoid these errors, it’s important to pay close attention to the sample size and the significance level in each experiment.

Article

4min read

Unleash your creativity: code once, customize infinitely

 

Say hello to Custom Widgets and goodbye to time-consuming back-and-forths when scaling ambitious customer experiences. With Custom Widgets, scale your best CX ideas across teams, brands and markets. AB Tasty has the largest widget library on the market, providing brands with over 25 pre-built ways to quickly engage consumers including scratch cards, NPS surveys and countdowns. But now we’re also giving you the ability to build, customize and share your own widgets! ?

Optimize the workflow between marketers, designers and developers

Custom Widgets are an innovation catalyst that fosters cross-team collaboration to bring ideas to life. Developers can now create highly customizable widgets following a step-by-step process. They simply code the different parts of the widgets using HTML, CSS and JavaScript and add various configuration options?‍?.  This allows designers to easily tailor the widgets and ensure they meet brand guidelines ?‍?.  Marketers can then customize them for their campaign needs ?‍♀️.The new possibilities to engage with visitors are endless: wheel of fortune, carousels, lightboxes, etc. These Custom Widgets result in an optimized workflow that saves everyone time but still delivers exciting experiences. ?

Create and scale a library of your best CX ideas

All Custom Widgets created (by developers, agencies, or AB Tasty) will be available in the widget library shared across all affiliates and accounts of a company. The library, accessible from the dashboard, is a great source of inspiration and ideation that will speed up time to market and facilitate deployment across brands and markets ✨. The widget library will also include our existing widgets with selected use cases from AB Tasty clients to further guide you in creating the best customer journey. And, like with any other widget, marketers can easily customize the content and combine it with AB Tasty’s targeting to create powerful personalized campaigns with no coding skills and in minutes ?‍♀️.

Not sure where to start?

In our new widget library, our users can already enjoy 2 custom widgets available on the platform, a Wheel of Fortune and a gradient CTA button, that they can duplicate and modify to dive into how they work. On that same page they can click on “Create a custom widget” and follow our step-by-step process ?. 

Why not try them now? If you’re looking for inspiration for your first Custom Widgets, check out our 30 Black Friday Tests ebook. It features successful tests from brands like Degrenne, a French cutlery and tableware retailer whose quality products are a staple in the hospitality industry. They wanted to accelerate the purchase process and provide a consistent omnichannel experience to their consumers. Using our widgets they gave their visitors the ability to see item availability in their local store ?.

If you want to replicate this, your developers can create a Custom Widget that leverages geolocation data to create a pop-up displaying product availability in nearby stores. Your customers will be able to reserve their items and opt for in-store pickup. Once available in the widget library, other brands or countries you work with can access it, modify it and leverage it to provide their visitors with an omnichannel experience.

To learn more check out the ebook ?:

With AB Tasty, let your good ideas take flight!

Article

5min read

1,000 Experiments Club: A Conversation With André Morys of konversionsKRAFT

André Morys shares the key to a thriving business in today’s market: a fearless attitude toward innovation and experimentation

André Morys has been involved in the field of experimentation for almost three decades. The CEO and founder of konversionsKRAFT discovered the importance of the user experience upon launching his e-commerce optimization company in 1996. He followed the evolution of high-end UX measurement solutions and AB testing throughout the 2000s, turning his focus entirely to experimentation and experience optimization in 2010.

Powered by his fascination for measuring the outcomes of the customer and user experience, including the financial impact of optimizing them, André uses his expertise to guide his clients through the experimentation process in a clear and efficient manner.

AB Tasty’s VP Marketing Marylin Montoya spoke with André about cultivating an experimentation mindset – that is, how to build a culture of experimentation that embraces failure and risk-taking – in order to innovate in competitive markets as an experimentation-driven company.

Here are some of the key takeaways from their conversation.

Risk and failure are integral components of innovation

Many risk-averse companies hesitate to implement experimentation due to their mindset around failure. When adopting a culture of experimentation, it’s essential to understand the fact that, despite prior research and preparation, failure is a part of the innovation and testing process. Most tests will produce negative results … but this shouldn’t be interpreted as failure.

Overcoming our personal beliefs and shame around failure and encouraging managers to lead by example will allow companies to embrace the experimentation process. This means discussing and deducing insights from failures within a team setting and allowing it to be a learning opportunity. Experimentation offers the chance for teams to evolve in how they see, feel and talk about failure and to focus on being solution-oriented rather than striving for perfectionism.

André also explains that while experimentation may seem risky, it is actually a way to control the level of risk linked to innovation. With experimentation comes feedback, allowing you to gauge the success of a particular feature or improvement, as opposed to naively rolling it out without any prior experimentation and simply hoping for the best.

Experimentation should always be linked to strategic goals and measured

The ROI of experimentation comes in many forms. André uses a “digital experimentation framework” to implement the experimentation process and measure progress. In relation to his “Iceberg Model”, while the tip of the iceberg represents the metrics that are measurable, many important factors lay beneath the surface and are difficult to measure, resulting in people underestimating the value of experimentation.

“You cannot measure the influence of experimentation towards your company’s culture and how big and…sustaining this effect will be. It will be maybe 10 times more important than the actual uplift you create,” says André.

Once discovering the process, many managers agree that a shift in culture, inventiveness and velocity is more powerful than an improvement in metrics. Meaning, there is a shift toward changing what’s below the surface (company culture) in order to have a bigger long-term impact, rather than focusing on the tip of the iceberg (metrics).

A frequent mistake that companies make is not aligning their goals of experimentation to their strategic goals, creating the possibility that their experimentation might be irrelevant. Using the experimentation framework to optimize the process with shared goals from the beginning will produce the best outcome.

Use gamification to engage teams and build a culture of experimentation

Overcoming resistance within companies to jump into experimentation doesn’t have to be difficult; in fact, it can even be fun!

Involving teams with guessing games and placing bets on which test won, for example, is a simple yet effective method to educate people and to create a pull towards experimentation. This technique also creates a sense of humility and allows people to realize that their hypotheses may fail and the customer could have a differing opinion.

Documenting the experimentation process and results is also important for encouraging cultural change. Connecting people to the outcomes enables learning and better engagement with the experimentation process, which in turn increases motivation to further improve. Rather than pushing people towards experimentation, you can create a natural attraction to it and a sense of personal investment by using metrics as feedback and creating a flywheel.

An experimentation dashboard is key to showing the health of your experimentation process. André highlights the importance of running effective tests, being sure to focus testing on the right areas, rather than striving for high velocity and volume of experiments. The goal should be experimentation quality, not quantity, and ensuring that important metrics are stored on a dashboard to facilitate cultural change.

1000 Experiments Club André Morys

What else can you learn from our conversation with André Morys

  • The three catalysts for innovation and why experimentation is a key component
  • How perfectionism blocks experimentation and the evolving experimentation culture in Germany
  • The status quo bias and why management might resist change within an organization
  • How the pandemic shifted companies’ focus towards digital growth
About André Morys

André Morys has almost three decades of experience in digital growth and business optimization, founding konversionsKRAFT in 1996. Today, his company is the leading business optimization consultancy in the DACH region, with more than 80 consultants, designers and conversion experts.

Specialized in research, consulting and lecturing, André combines qualitative research, consumer psychology and behavioral economics with data and experimentation to deliver the most effective growth programs for enterprise clients.

André’s reading recommendations:
About 1,000 Experiments Club

The 1,000 Experiments Club is an AB Tasty-produced podcast hosted by Marylin Montoya, VP of Marketing at AB Tasty. Join Marylin and the Marketing team as they sit down with the most knowledgeable experts in the world of experimentation to uncover their insights on what it takes to build and run successful experimentation programs.

Article

7min read

How to Deliver “Wow” Digital Customer Experiences

Customer centricity might be one of the buzzwords of business today, but the demand for good service is hardly new. So why does it suddenly matter so much nowadays?

In our rapidly-changing, technology-driven, multichannel society, the product itself is no longer the sole reason for a customer to interact with your brand. The entirety of the purchasing process is crucial, as it shapes repeat purchases and customer loyalty, lifetime customer value and broader engagement with your brand. As we’ve seen when discussing the experience economy, experience matters – and if you can create this layer of value beyond the mere acquisition of your product, you can get ahead of the game.

Multichannel means both digital and physical touchpoints: e-commerce stores alongside physical stores, AI-driven service tools alongside customer call centers. Mastering all of these is essential, but where in-person experiences can be managed on an individual and human-controlled basis, digital experiences rely on technology in the back end and the user journey mapping your team has established when building your website and mobile app.

Given the autonomy with which customers approach digital interactions with your brand, ensuring your online channels are operating smoothly, the transaction and checkout process is clear and bug-free, the page load times are imperceptible, and key information like shipping and returns policies are readily available builds the bare-minimum experience that customers now expect. The next step is to ramp it up and add the “wow” factor.

Customers now expect “wow” experiences when shopping both online and in-person
Customers now expect “wow” experiences when shopping both online and in-person (Source)

A great way to knock the socks off your customers is by personalizing their experience, leveraging the power of data to curate an exclusive and relevant interaction for them each time they visit your site. Think about tailored product recommendations and discount offers or content that’s targeted to each visitor’s unique desires. If you nail personalization, you’ll also go a long way to nailing retention and repeat business: according to a report by Accenture, 91% of consumers are more likely to buy from companies that remember them and provide relevant offers. Not convinced? On the flip side, 74% of customers get frustrated when content has nothing to do with them. Is that a risk you’re willing to run when it comes to how your brand is perceived? Bottom line, bring the “wow” or your customers will go elsewhere.

How to go from functional to “wow” customer experiences

In previous posts, we’ve explored the notion of experience optimization, and that’s your starting point when it comes to customer experience. Here we’re referring to smooth, glitch-free transactions on functional websites with clear CTAs that drive conversion. It’s the bare minimum required in the e-commerce space today, and it’s driven by a continuous feedback loop founded on A/B testing and leveraging customer data to ensure your platform is always optimized.

The bare minimum might keep your business viable, but if you want to go to the next level, experience innovation is the next step toward CX that makes your consumers go wow. Competition in the online shopping space is intense and the leaders secure their advantage by investing in innovation and going beyond the basic transactional elements of product acquisition.

Experience innovation is key for wowing your customers
Experience innovation is key for wowing your customers (Source)

Put optimization and innovation together and what do you get? That “wow” experience that puts you ahead of your the competition and keeps customers coming back for more. It’s an improve, adapt, anticipate process flow that typifies this dynamic:

  1. Improve: Start with basic UX improvements to your website to increase your conversions.
  2. Adapt: Learn and streamline your process so that the team can focus on efficiency and what drives value.
  3. Anticipate: Surpass your customers’ expectations by creating agility in your optimization, experimentation and innovation approaches that enable creativity and sparkling results.

Being meaningful and memorable seems like a no-brainer of a strategy, right? Personalize, optimize and innovate, and you’ll be top of mind in your category and driving conversion and revenue all the way to the bank.

Three keys for ramping up your customer experience

1. Data-powered customer insights

Knowing and understanding your customers is essential to recognizing – and eventually anticipating – their needs. The best approach to building this understanding is through a solid data foundation that will allow you to collect and analyze data about customer behaviors and trends. Customers trust brands that protect their personal data, so being transparent about data collection and use, and setting up a robust approach to collating and processing that data, will ensure that you retain that all-important trust factor while building a workable infrastructure that serves your optimization and experimentation roadmap. From here, you can understand customer intent: what they’re looking for, what their needs and wants are, and how you can tailor your offering to meet and exceed these expectations. When you get to the point where you know what your customers want before they realize it themselves, you’ll be poised to deliver that next-level experience.

2. Create a personalized experience

Your data foundation will also drive your personalization efforts and help you go far beyond that name in a direct email. Given 48% of consumers spend more when their experience is personalized, this is another sure-fire strategy to knock your CX out of the park and deliver returns to your business as well. UK-based beauty retailer Fragrance Direct used personalization to great effect during the holiday period, designing a “Fragrance Finder” tool that provided tailored product suggestions in response to a series of questions about the user’s needs. By then targeting site visitors with a “Searching for a gift?” CTA, the brand leveraged high-traffic shopping periods to both fulfill their customers needs and optimize their experience. This personalized approach delivered a 10% increase in user conversion during peak shopping periods alongside an exceptional experience for Fragrance Direct’s customers.

3. Deliver superior interactions through omnichannel coverage

Omnichannel strategy comprises brand touchpoints that cover both in-person, in-store interactions and online web, mobile and social retailing and exchanges that are powered by data, tools and AI. According to Harvard Business Review (HBR), this strategy is built on the notion that shopping experiences across multiple touchpoints – both brick-and-mortar and online stores – not only differentiates retailers from their peers but gives them an advantage over single-channel businesses. Enabling your customers to engage with you whenever and however they feel like delivers better returns. By powering these experiences with tools like chatbots, self-service and human-driven help centers, you can keep customers coming back again and again.

Fuel customer loyalty by optimizing shopping experiences across multiple touchpoints
Fuel customer loyalty by optimizing shopping experiences across multiple touchpoints (Source)

Take the example of OUI.sncf, the European distribution channel of the French railway network, whose team used the AI-powered exploration assistant to support travelers’ itinerary planning and provide personalized suggestions for those with high interest scores. The tool not only delivered support to site users when ticket counters were closed, it also drove a 61% increase in web responsive and a 33% increase in desktop conversion rates for online shoppers.

Innovate and optimize your omnichannel strategy

To be completely blunt: Customers don’t care about the tiny details of your digital process and platform until those details impact their experience. It’s the experience that wows them and keeps them coming back again and again, so building out an omnichannel strategy around brand interactions should be the core of your focus. A study from HBR found that omnichannel customers represent more value for businesses: they spend 4% more in-store and 10% more online than single-channel shoppers. From website to mobile and social engagements, to your customer support centers and in-store, in-person connections, seamlessness and continuous “wow” moments will be what sets you apart from others in your category.

Consistency also relies on constant optimization of your digital interfaces – website, mobile app, social touch points – by adopting a continuous testing approach driven by your experimentation roadmap. This perspective ensures that nothing gets left on the table from a conversion standpoint and also that customers are consistently blown away by their time spent engaging with your brand. What do you get when you put together experience optimization and experience innovation? Digital experience mastery.

Looking for more on delivering impactful brand experiences that will wow your customers? Check out AB Tasty’s digital customer journey, a roadmap to driving optimization and innovation that will take customer experience for your brand to the next level.

Article

9min read

What Is Customer Experience Innovation?

In today’s world of fickle attention spans and abundant choice for consumers, building your brand experience is no longer something that companies could consider doing. Nowadays, it has become a must-have for anyone that wants to stay in the game.

Establishing a relationship with your customers by adding value to each touchpoint – be it via services that go beyond what they’d expect, special rewards that inspire and entice, or personal touches that address their direct needs – will be what keeps them coming back. It will also play an integral role in building your brand’s very reputation by delivering the ‘wow’ experiences that put you ahead of your competition and at the top of the class.

In recent weeks we’ve discussed the importance of customer experience optimization (that is, maximizing conversion and delivering against KPIs, as well as leveraging responsive and quick-win experimentation to ensure nothing gets left on the table). When it comes to customer experience innovation, it’s about taking that to the next level. If optimization is the bare minimum that you should be doing, innovation is maximizing the long-term value of your brand and building a competitive edge to set you apart from the other brands in your category.

In this article, we’ll cover:

[toc]

Why customer experience innovation matters

Here at AB Tasty, customer experience innovation means going beyond the product to create an exchange that delights your customers, cements their loyalty and sets the bar so high that you’re the standard they come to expect from every company they encounter. It’s also more than just optimizing to ensure you have a high-performing, functional website. When it comes to innovation, the goal is to stand out from the pack, staying ahead of your competition, to create a signature brand experience that distinguishes your business from others.

Think of Spotify. At a basic level, they’re a streaming service that offers a huge library of content that is easy to access, and simple to subscribe to with seamless payment that makes for uninterrupted listening. But Spotify is more than just an optimized service, they’re also about innovation that delivers experiences that go beyond. One example is their user recaps, which leverage data to create a personalized experience to help listeners celebrate who they are (based on what they’ve listened to) and give each individual their own story to tell. (It’s also a nifty way for Spotify to get their users to advertise on their behalf!) Spotify is also making it clear that they’re more than a streaming service: they’re the embodiment of their users’ wishes brought to life.

Spotify leverages data to create a personalized experience for their listeners
Spotify leverages data to create a personalized experience for its listeners (Source)

Companies that wrap an immersive experience around their product (as Spotify does with its year-end recaps) create a more engaging environment for their consumers that goes beyond the mere items they sell and delivers an experience that’s more than just a transaction. From Nike creating a community of fitness to Tim Horton’s gamifying its loyalty program and Oui.SNCF leveraging AI to elevate trip planning, these companies are using customer experience innovation to drive sales.

The key components of experience innovation

In 2020, Accenture’s Business of Experience report found that 77% of CEOs believe their company will fundamentally change the way it connects and interacts with its customers, and that leading companies are twice as likely to have the agility to pivot towards new models that deliver value than their competitors. Not sure on which side you fall? Let’s take a look at the key elements for customer experience innovation.

In the current environment of fast-moving technological change marked by devices and services which are never far out of reach (and thus never truly off), your brand is accessible at all times; long gone are the days when shops would close and your customers would have to wait for them to reopen the following morning. This presents a multitude of opportunities to drive meaningful interactions and engagements with your consumers alongside added value to your business. And to get there, you need to leverage experimentation.

Companies need to leverage experimentation in order to drive meaningful interactions with their consumers
Companies need to leverage experimentation in order to drive meaningful interactions with their consumers (Source)

Experimentation can be run client-side (front-facing, on the website’s interface) and server-side (on the back-end, across all digital touchpoints if necessary). Client-side testing runs in your visitors’ browsers and is limited in scope to largely aesthetic and layout measures. To dive deeper into experience innovation, you’ll need to get into server-side.

Server-side experimentation

Server-side experimentation allows for more sophisticated experiments, tests features that go beyond the surface level and is platform- and language-agnostic. It’s also a heavier lift and needs developer and tech team input; as it’s run using a website’s source code, this testing relies upon coding skills. To implement server-side experimentation, you’ll need buy-in from both marketing and product teams, and a willingness to invest developer resources into running your experiments. But you’ll also achieve more flexible and sophisticated testing, such as price sensitivity and elasticity testing, as well as testing across multiple channels.

Feature management

Feature management is a process by which developers release updates gradually, through the use of feature flags, to allow platform updates to be tested while minimizing the risk of major site crashes or performance issues when rolling out new software releases. Using progressive deployment and rollbacks, where parts of the code are removed to allow features to be toggled off and on, feature management can test multiple versions of an update to determine which yields the best result – optimizing against set KPIs – and should thus be adopted permanently. Using this approach also ensures that you nail the transition to an updated platform with existing users, delivering an elevated experience that guarantees they never look back.

Each experimentation method has strengths and challenges, but it is in their combination that their greatest power lies. By leveraging both client- and server-side testing, you are able to go beyond optimization to build total brand experiences.

Get the most out of experimentation by leveraging both client- and server-side testing
Get the most out of experimentation by leveraging both client- and server-side testing (Source)

Three innovative companies that are taking up the challenge

1. Zwift

Zwift is a multiplayer online game and fitness platform that leverages virtual reality to transport its players’ running and cycling workouts to various iconic locations around the world. Ever wanted to tackle the famous Alpe d’Huez stage of the Tour de France or the bone-jamming cobblestones of Paris-Roubaix? This is the kind of platform that can make that happen. Users connect their turbo trainer or treadmill to the Zwift app and the in-game avatars bring workouts to more than 240 miles of virtual terrain, and permit group sessions and participatory events such as the Virtual Tour de France. The pandemic saw a considerable upswing in at-home fitness, but Zwift’s innovation takes the experience of working out at home to another level.

Zwift takes the experience of working out at home to another level with virtual reality
Zwift takes the experience of working out at home to another level with virtual reality (Source)

2. Uber

Ride-sharing phenomenon Uber identified that 60% of trips in Sydney, Australia, begin or end in areas with limited access to frequent public transport. Leveraging that user insight, they launched the Uber and Transit feature in September 2020, enabling riders to identify the best combination of public transit and UberX rides to complete their journey. The feature gives passengers the ability to compare the cost and time for their trips depending on the constellation of transport methods they adopt, an approach that prioritizes customer needs without driving users away from their core service.

Uber prioritizes customer needs to offer them a better experience
Uber prioritizes customer needs to offer them a better experience (Source)

3. On

Consumers are increasingly conscious of the sustainability commitments of the brands with which they engage. Swiss sporting goods manufacturer On adopted a subscriber approach to support a business model which encourages circularity without stymying both the desire and the necessity to consume products (in this company’s case, shoes). Customers pay a US29.99 subscription fee which allows them to swap out their current shoes for new ones as often as they’d like, and also delivers On sufficient sneaker returns to make circularity feasible. The shoes are made from castor beans and can be completely recycled, giving that growing consumer base of sustainability-focused customers peace of mind whilst still serving their performance needs.

On offers its sustainability-focused customers a product that caters to their needs
On offers its sustainability-focused customers a product that caters to their needs (Source)

Want even more best-practice examples of brands hitting it out of the park? Check out AB Tasty’s guide to optimization trends. Get your copy of the “50 Tests You Should Know for Your Website” E-book now!

Collaborate across teams for continual evolution and development

We’ve already established that experience optimization is the bare minimum when approaching your brand’s online presence and commercial activities, and that experience innovation is what takes you to the next level in your category. To innovate is to experiment – exploring different configurations, layouts, price thresholds and incentives, as Jonny Longden of Journey Further told us on the “1000 Experiments Club” podcast. Your experimentation roadmap is essential to retaining your customers, recruiting new ones and growing your business.

Experience innovation is not owned by one team: It takes multiple divisions collaborating toward the common goal that is established by your roadmap. Setting up your internal organization to anticipate customer demands requires investment in your tech stack, alignment and cooperation between product, tech and marketing teams, and allocation of resources in accordance with your agreed-upon experimentation plan.

Experience innovation requires alignment between product, tech and marketing teams
Experience innovation requires alignment between product, tech and marketing teams (Source)

To maximize customer experience innovation, your teams should be empowered to be the innovators. Allocate resources and responsibilities fairly and toward efforts that the individual teams can influence, simplify the tech processes for implementation and rollout, and drive innovation around business priorities so that everyone is paddling in the same direction and the outcomes from experimentation efforts find success.

Article

5min read

1,000 Experiments Club: A Conversation With Jeremy Epperson of ConversionAdvocates

Jeremy Epperson explains why startups should leverage conversion rate optimization to maximize growth.

Jeremy Epperson is about to change the way you approach growth in your business. The chief growth officer at ConversionAdvocates, a top-ranking CRO agency specialized in data analysis, takes a data-driven approach to identify the roadblocks in testing and optimize these processes for maximum effectiveness.

Over the past decade, he has launched CRO programs for 150+ growth-stage startups, creating a repeatable proven process for conversion rate optimization that can be implemented across different verticals and business sizes. By collating the insights gained from the different businesses, notably the common mistakes, Jeremy has gathered the expertise to facilitate CRO programs and avoid the steep learning curve that comes with launches.

In his conversation with AB Tasty’s VP Marketing Marylin Montoya, Jeremy delves into the granular level of data analysis and takes on topics that most people in CRO steer clear of.

Focus on customer experience optimization to catapult business growth

In today’s digital landscape, the old-school ideology of branding and push marketing is no longer an effective strategy. These days, customers have easy access to online reviews, forums and price comparison websites to inform their purchasing decision.

Rather than trying to control the customer journey, Jeremy recommends optimizing the experience of each of its four phases, using a data-driven, scientific-testing approach. This leads to the creation of different processes and reshapes the idea of optimization: The game-changing idea is that agility (allowing companies to move, learn and improve faster) can trump exorbitant budgets, thus allowing smaller companies to take market share from giants.

Passionate about being involved with teams on the ground level to “iteratively work through the entire process,” Jeremy touts CRO as the best mechanism and catalyst for growth, which challenges teams to rethink and rebuild processes and workflows, break down silos and build communication. Jeremy says this team-building aspect is more valuable from a CRO perspective than any individual winning test.

All data is equal: the value of wins, losses and flat tests in post-test analysis

When it comes to testing, certain results are deemed more “sexy” by marketers, and others are often swept under the carpet. However, Jeremy explains the utility of all test results, be that a win, a loss or a flat result, for informing how testing should evolve.

A string of inconclusive tests means that the testing has not been focused on what is actually blocking the conversion. “If we’re not targeted in on the things that are blocking them (users) from converting then we’re not going to see big movement in the conversion rates, so that’s really important,” says Jeremy.

When test results show big changes in the conversion rate, positive or negative, this indicates that an important part of the customer experience has been impacted. While winning tests are celebrated and losing tests shied away from, Jeremy advises that in both cases, the next step should be to double down on test variations to fully resolve the problem, creating at least three variations for each of those hypotheses.

Understand your customer and remove their purchasing roadblocks

Oftentimes, marketers, especially in smaller businesses, are reluctant to spend their budget on research and insights, opting for customer acquisition strategies involving ads and content. However, according to Jeremy, investing in research to better understand the customer can bring us closer to answering one question that’s key to creating the right growth strategy for your business: Why does your customer buy or not buy your product?

Research and testing can offer 360-degree insights into customer behavior such as their buying criteria, decision-making and their buying process in order to remove any conversion roadblocks. It could be as simple as creating an FAQ page to clarify primary questions, resulting in a 23% lift in lead conversion, as Jeremy exemplified.

Jeremy explains that businesses will naturally experience growth when they focus on offering a better customer experience, eliminating customer frustrations and roadblocks, which would otherwise cause them to abandon their purchase. This customer-centric mindset will actually have a direct positive impact on revenue and growth.

Jeremy Epperson

What else can you learn from our conversation with Jeremy Epperson

  • How to combine research and testing in CRO to double the average validated win rate
  • How to encourage teams to embrace the CRO process and cooperate across verticals
  • The inutility of customer personas and how to replace them
  • How to implement CRO for the first time
About Jeremy Epperson

Jeremy Epperson, chief growth officer at ConversionAdvocates, has worked in the field of startup growth and conversion rate optimization (CRO) for 14 years, as a consultant in his own businesses as well as part of digital agencies. Jeremy is passionate about researching, building and implementing processes to generate growth and has launched CRO processes within more than 155 growth-stage startups. He also specializes in customer journey mapping, CRO maturity assessments and marketing and customer research.

About 1,000 Experiments Club

The 1,000 Experiments Club is an AB Tasty-produced podcast hosted by Marylin Montoya, VP of Marketing at AB Tasty. Join Marylin and the Marketing team as they sit down with the most knowledgeable experts in the world of experimentation to uncover their insights on what it takes to build and run successful experimentation programs.

Article

9min read

How to Create a Modern Data Foundation for Experimentation

Staying ahead of the game to deliver seamless brand experiences for your customers is crucial in today’s experience economy. Today we’ll dip our toe into the “how” by looking at the underlying foundation upon which all of your experiences, optimization and experimentation efforts will be built: data.

Data is the foundation experimentation is built on
Data is the foundation experimentation is built on (Source)

 

Data is the technology that can power the experiences you build for your customers by first understanding what they want and how it’ll best serve your business to deliver this. It’s the special sauce that helps connect the dots between your interpretation of existing information and trends, and the outcomes that you hypothesize will address customer needs (and grow revenue).

If you’ve ever wondered whether the benefits of a special offer are sufficiently enticing for your customer or why you have so many page hits and so few purchases, then you’ve asked the questions the marketing teams of your competitors are both asking and actively working to answer. Data and experimentation will help you take your website to the next level, better understand your customers’ preferences, and optimize their purchasing journey to drive stronger business outcomes.

So, the question remains: Where do you start? In the case of e-commerce, A/B testing is a great way to use data to test hypotheses and make decisions based on information rather than opinions.

A/B testing helps brands make decisions based on data (Source)
A/B testing helps brands make decisions based on data (Source)

 

“The idea behind experimentation is that you should be testing things and proving the value of things before seriously investing in them,” says Jonny Longden, head of the conversion division at agency Journey Further. “By experimenting…you only do the things that work and so you’ve already proven [what] will deliver value.”

Knowing and understanding your data foundation is the platform upon which you’ll build your knowledge base and your experimentation roadmap. Read on to discover the key considerations to bear in mind when establishing this foundation.

 

Five things to consider when building your data foundation

  1. Know what data you’re collecting and why
    Knowing what you’re dealing with when it comes to slicing and dicing your data also requires that you understand the basic types and properties of the information to which you have access. Firstly, let’s look at the different types of data:

    • First-party data is collected directly from customers, site visitors and followers, making it specific to your products, consumers and operations.
    • Second-party data is collected by a secondary party outside of your company or your customers. It’s usually obtained through data-sharing agreements between companies willing to collaborate.
    • Third-party data is collected by entirely separate organizations with no consideration for your market or customers; however, it does allow you to draw on increased data points to broaden general understanding.

     

    Data also has different properties or defining characteristics: demographic data tells you who, behavioral data tells you how, transactional data tells you what, and psychographic data tells you why. Want to learn more? Download our e-book, “The Ultimate Personalization Guide”!

    Ultimate personalization guide e-book

    ‎‎ㅤ‎‎

    Gathering and collating a mix of this data will then allow you to segment your audience and flesh out a picture of who your customers are and how to meet their needs, joining the dots between customer behavior and preferences, website UX and the buyer journey.

    Chad Sanderson, head of product – data platform at Convoy, recommends making metrics your allies to ensure data collection and analysis are synchronized. Knowing what your business leaders care about, and which metrics will move the business forward, will ensure that your data foundation is relevant and set up for success.

    ‎ㅤ

  2. Invest in your data infrastructure
    Data is everywhere, in its myriad of forms and gathered from a multitude of sources. Even so, if you’re going to make use of it, you need a robust system for gathering, storing and analyzing it in order to best put it to work. Start by understanding how much first-party data you have the capacity to gather by evaluating your current digital traffic levels. How many people are visiting your site or your app? You can get this information using Google Analytics or a similar platform, and this will help you understand how sophisticated your data-leveraging practices can be and identify gaps where you might need to source supplementary data (second- and third-party).
    Next, you’ll need to evaluate your infrastructure. Companies that are further on their data analytics journey will invest in customer data platforms (CDPs) that allow them to collect and analyze data – gathered from a variety of sources and consolidated into a central database – at a more granular level. Stitching together this data via a CDP helps you bring all the pieces together to form a complete picture of your customers and identify any gaps. This is a critical step before you leap into action. Chad Sanderson concurs. “[Start] with the business and what the business needs,” he advises. “Tailoring your… solution to that – whatever that is – is going to be a lot more effective.”‎
  3. Get consent to build consumer trust
    Data security is rightly of foremost concern to consumers. The very users from whom you want to gather that first-party data want to ensure that their private information remains secure. Getting their consent and being transparent about the inherent benefit to them if they agree to your request – be it through giveaways, exclusive offers, additional information or services – will give you the best chance of success. Demonstrating that you adhere to, and take seriously, various data compliance laws (such as GDPR) and good governance will also build trust in your brand and give you the opportunity to make it worth their while through improved UX and personalized experiences.

    Build trust in your brand by respecting your users’ private information
    Build trust in your brand by respecting your users’ private information (Source)

    ‎ㅤ

  4. Collect and discover insights to upgrade your customer strategy
    We’ve already covered the fact that data is everywhere. As Chad Sanderson highlighted above, identifying immediate business needs and priorities – as well as focusing on quick wins and low-lift changes that can have a quick and high-level impact – can help you navigate through this minefield. It’s best to think of this section as a four-step process:
    ㅤㅤ Collect data as it flows into your CDP
    ㅤㅤ• Transform or calibrate your data so that it can be compared in a
    ㅤ  ㅤlogical manner
    ㅤㅤ• Analyze the data by grouping and categorizing it according to
    ㅤ  ㅤthe customer segments you’ve identified and benchmarking
    ㅤ  ㅤagainst business priorities
    ㅤㅤ• Activate your insights by pushing the learnings back into
    ㅤ  ㅤyour platforms and/or your experimentation roadmap and really
    ㅤ  ㅤput this data to work
  5. Turn your data into actions
    It’s crunch time (no pun about numbers intended)! We’ve examined the different types of data and where to source them, how to be responsible with data collection and how to set up the infrastructure needed to consolidate data and generate insights. We’ve also covered the need to understand business priorities and core strategy to drive data collection, analysis and activation in the same direction. Now we need to put that data and those insights to work.
    In the experience economy, where constant evolution is the name of the game, innovation and optimization are the key drivers of experimentation. Taking the data foundation that you’ve built and using it to fuel and nourish your experimentation roadmap will ensure that none of the hard work of your tech, marketing and product teams is in vain. Testing allows you to evaluate alternatives in real time and make data-driven decisions about website UX. It also ensures that business metrics are never far from reach, where conversion and revenue growth take center stage.Use the data you’ve gathered to fuel your experimentation roadmap
    Use the data you’ve gathered to fuel your experimentation roadmap (Source)

 

Invest in a solid data foundation to maximize and scale

At AB Tasty, we apply the Bayesian approach to interpreting data and test results because in A/B testing, this method not only shows whether there is a difference between the tested options but also goes beyond that by calculating a measure of that difference. Being able to identify what that variance is allows you to best understand what you will gain by adopting a permanent change.

Collecting and analyzing data, and then leveraging the insights that you glean, are key to unlocking the next level of experience optimization for your customers and your business. An experimentation roadmap grounded in real-time responsiveness and long-term, server-side improvements will have a solid data foundation approach at its core, where understanding who you want to target and how to act drives success. Furthermore, if you invest in your data foundation – and the five core drivers we’ve explored above – you’ll be equipped to scale your experimentation and allow optimization to become a key business maximizer.