Bayesian Statistics — An Introduction

As an avid enthusiast of Bayesian statistics, I’m often reminded of the Oxford comma — both are beloved by a passionate few who are willing to embrace their eccentricity, even if it means being seen as a mild outcast in respectable society. In this article series, I hope to share my love of Bayesian statistics and show you how this approach can expand your understanding of statistics, data science, and even the world at large. (And yes, I might have included that last point just to sneak in an Oxford comma.)

So what sets Bayesian statistics apart from “normal” statistics (also known as frequentist statistics)? The key difference is in how we use probability distributions to make sense of historical data. In frequentist statistics, we rely on statistical laws and theorems to substitute a probability distribution for the data we have. Typically, this means reducing the data to a few key summary statistics, like the mean and variance.

In contrast, Bayesian statistics eschews these broader statistical laws. Instead, we start by making an initial assumption about what the correct probability distribution might be, and then update that assumption as we incorporate historical data. This “updating” process transforms our initial assumption (called the “prior distribution”) into a final distribution (the “posterior distribution”). The result is a beautiful, self-contained system in which each historical data point is incorporated into the posterior distribution without relying on external statistical theory or reducing the data…