Over the past few decades, we have witnessed a quiet revolution in the understanding of probability and, with it, the production of knowledge. One could call it the Bayesian revolution, after the 18th century statistician Thomas Bayes, who proposed that probability represents not an objective assessment of an event’s frequency but a subjective measurement of belief: an informed prediction about whether an event will occur.
While relatively simple mathematically, Bayesian statistics is computationally intensive, which, until recently, made it prohibitively difficult to operationalize. But with the development of modern computational capacity, Bayesian probability is ascendant, offering a way to turn the large volumes of data now being captured into predictions and “insights.” It provides the underlying logic for Google’s search engine, Five Thirty-Eight’s mode of political coverage, and algorithmic trading in financial markets. Most everything that we think of as artificial intelligence is also a product of applying statistics and probability to a growing field of problems — everything from predictive policing to online dating to surveillance assessment to targeted ads to voice detection and interpretation. Any time you see a moving needle assessing an election night outcome, use a spam filter, or receive a recommendation based on what others “like you” have done, you are experiencing the products of Bayesian probability in action.
While the Taylorist and Fordist revolutions created the conditions for the “automatic” production of goods,the Bayesian revolution is cementing the conditions for the automatic production of knowledge
Bayesian thinking has been hailed by a handful of technologists and scientists as a panacea for our broken times. Countless articles (including this one in Aeon and this peer-reviewed article in PLOS ONE about replication problems in psychology) have suggested that Bayesian statistics could, as though by magic, cure the replication crises that have plagued various scientific disciplines, addressing the corrupting influences of incentives. The Bayesian approach has even inspired a self-help movement of sorts devoted to promoting “rational thinking,” published under the moniker Less Wrong. Bayesian thinking has become idealized as a norm to aspire to and ultimately to be controlled by....MORE
