Bayesian statistics is a subject with an endless level of depth and complexity. Navigating the many facets of Bayesian theory can prove challenging, but with the right foundations, it is more than possible.
A branch of statistics that is challenging and rewarding in equal measure, Bayesian statistics offers a wealth of benefits to those that commit.
To help you on your journey toward analytical enlightenment, here is everything you need to know to get started with Bayesian statistics, and more importantly, how you can apply it to your A/B tests.
Bayesian Statistics: A Definition
Before we delve any deeper into this all-encompassing analytical field, it’s important to ask the question: What is Bayesian statistics? Here’s an official definition for your reference, according to Scholarpedia:
“A system for describing epistemological uncertainty using the mathematical language of probability. In the ‘Bayesian paradigm,’ degrees of belief in states of nature are specified; these are non-negative, and the total belief in all states of nature is fixed to be one. Bayesian statistical methods start with existing ‘prior’ beliefs, and update these using data to give ‘posterior’ beliefs, which may be used as the basis for inferential decisions.”
In slightly simpler terms, Bayesian statistics is a concept based on belief and probability.
It’s a statistical model used to determine a belief and its possible reality mathematically—and it’s a theory you can apply to many aspects of life or the world around you.
By applying probability to a statistical problem, it’s possible to quantify or refine our beliefs around a number of random occurrences or events by analyzing relevant statistics or data.
Understanding Bayes’ Theorem and Prior Probability
When getting to grips with a certain concept or theory, it’s important to understand where it originated. Bayesian statistics derived from British mathematician, Thomas Bayes.
Thomas Bayes is responsible for Bayes’ Theorem, the foundations on which Bayesian statistics stand. The fundamental component of Bayesian statistics, Bayes’ Theorem, is based on probability theory, the branch of statistics focused on the analysis of random events.
Another key element of the Bayesian way is a little something called prior probability. In Bayes’ Theorem and Bayesian statistical models, prior probability represents the belief, or the uncertain aspect or quantity, before you take analytical evidence into account.
We might have a prior belief about an event or occurrence, but our perceptions are more likely to change when new evidence is presented. Bayesian statistics offers us a dependable mathematical method of merging our prior beliefs, and evidence, to produce brand new posterior views, beliefs, or ideas. This, in essence, is the Bayesian way.
Adopting the Bayesian Mindset
To get under the skin of Bayesian statistics and Bayesian inference, adopting a Bayesian mindset is a must. That said, here we explore two common probability scenarios and how the Bayesian mindset applies to these problems.
An Unfair Coin Toss
Consider an unfair coin toss where the coin in question may be manipulated or modified. In this instance, the individual in question (essentially, the victim of the unfair coin toss) may believe that the coin is fair or regular. But, if after five or six flips, the coin continually lands on heads, one’s prior beliefs may be modified—and thus, it could be possible that the coin is unfair.
After perhaps 300 to 400 tries, the individual adopts the posterior belief that the coin is incredibly likely to be rigged or manipulated. Thus, the posterior belief is strikingly altered from the prior belief that the coin is fair.
A Formal Election
In an election, a voter has the prior belief based on a particular candidate’s chances of winning, an instance, or outcome that it’s possible to quantify as a mathematical probability.
But another voter could hold a separate prior belief based on the same candidate’s likelihood of winning the election. However, as new metrics, information, and data related to the election in question emerge, both beliefs will become rationally and statistically transformed or altered. Again, the posterior belief will become removed from each prior belief, and further quantified by Bayesian methods.
At its core, the Bayesian mindset is based on individual outlook or opinion. Different people often have individual prior beliefs concerning an event or situation, depending on the information or data available to them. As new data presents itself, these prior beliefs transform and are likely to converge, resulting in posterior beliefs that are far more synergistic or agreeable.
Practical Applications of Bayesian Statistics
Now that we’re better acquainted with the fundamental concepts and mindsets surrounding Bayesian statistics, we can continue our journey by looking at its practical applications.
After pondering the question ‘what is Bayesian statistics?’ it’s only natural to wonder what Bayesian statistics are used for in a practical capacity.
While sophisticated predictive innovations—largely powered by machine learning (ML) technology—exist to formulate autonomous algorithms to process data and project particular outcomes, such tools have their limitations. The primary setback of machine learning-based statistical algorithms is the fact that they rely on large datasets to return data or insights that are reliable.
There is little room for inaccuracy when it comes to predictive risk assessment, and so, in many cases, Bayesian inference is applied to provide a number of potential outcomes based on solid statistical foundations.
Bayesian statistics is typically used in situations where data is muddy or noisy. By applying Bayesian analytical techniques, you can gain clarity on these areas of uncertainty and arrive at informed conclusions or potential outcomes.
In addition to patient care and financial forecasting, Bayesian statistics is also utilized practically in the following sectors, among others:
- Marketing research: Across sectors, data-driven marketing experts use Bayesian approaches to project what consumers may or may not respond to in terms of content or user experience (UX). This offers the power to develop strategies that increase engagement and meet the needs of a specific target audience while making informed decisions that save time and money, while reducing the potential for misguided or redundant marketing activities.
- Digital marketing analytics: In all facets of digital marketing, data, if leveraged to its fullest potential, translates to success. Digital marketing analytics is no exception. By using Bayes’ Theorem to outline the probability of consumer behaviors or outcomes based on particular conditions or datasets, it’s possible to create predictive analytical models that help marketers better meet the needs of their audience while developing campaigns that boost awareness while increasing conversions.
- A/B testing: Another key practical application of digital marketing, it’s possible to use Bayesian statistics to improve the A/B testing process. The comparative nature of A/B testing (comparing two versions of a landing page to see which one drives the greatest level of engagement, for instance) lends itself to the Bayesian model. As such, it’s possible to use Bayesian inference models to drill down into existing knowledge, metrics, and potential outcomes for improved success—as outlined in this study by statistical expert Roopam Upadhyay.
Why We Use Bayesian Statistics for A/B Testing
At AB Tasty, we use Bayesian statistics to meet “marketers’ and online business owners’ needs for immediate access to information and fast decision-making while ensuring the reliability of the results.”
Until recently, the most common way to run A/B tests was to use frequentist statistics. The issue with that method, however, is that it tests a hypothesis without having given it a probability.
Instead, Bayesian A/B testing assigns a probability to a hypothesis. In other words, Bayesian theory determines the probability that a hypothesis is true or false (e.g. whether A is different from B).
Additionally, Bayesian theory takes into account past data which is fed into the prior probability to inform the posterior probability. Combining the prior and posterior data enables us to make a conclusion about the result of the A/B test.
While the results may be similar to a test run by frequentist methods, there are some advantages of using Bayesian statistics:
- You can start a test without having a defined sample size beforehand. Instead, you simply start your test and as soon as a significant result is identified, you can use it.
- The main advantage of using bayesian analysis is that you get a confidence interval around the gain measure. This helps you to figure out a best and worst case scenario. Frequentist analysis only provides a probability to have a positive gain, but it provides nothing about the estimation of its size. Bayesian analysis does, and it make a huge difference when dealing with business decisions.
This means that although the Bayesian approach is more complex than the frequentist approach, it is more reliable. The increased reliability is due to the assumption that if a certain outcome has been observed before, there’s a greater chance that it’ll happen again in the future.
The Final Word
It’s clear that aside from being somewhat of academic curiosity, Bayesian statistics has emerged as an invaluable tool across industries, particularly in A/B testing, website development, and digital marketing.
Its applications are broad, and by getting a well-rounded working knowledge of Bayesian techniques, there is a wealth of insight to gain.
Bayesian statistics is a challenging yet rewarding subject. Use this guide and its suggested resources to get started and you will gain the knowledge base required to take your Bayesian studies to the next level. Good luck!