Saturday, May 24, 2025

"Everything Is Correlated"

Which is very different from the ecologist saying everything is connected.

Correlation is sometimes just an artifact of the human propensity to impose patterns on what we sense. Connections often exist independent of the observer. And can be very profitable. Just ask James Burke.*
(and yes, "sometimes" and "often" are doing a lot of the heavy lifting in that paragraph) 

From Gwern  2014-09-128y2023-04-30: 

Anthology of sociology, statistical, or psychological papers discussing the observation that all real-world variables have non-zero correlations and the implications for statistical theory such as ‘null hypothesis testing’.

Statistical folklore asserts that “everything is correlated”: in any real-world dataset, most or all measured variables will have non-zero correlations, even between variables which appear to be completely independent of each other, and that these correlations are not merely sampling error flukes but will appear in large-scale datasets to arbitrarily designated levels of statistical-significance or posterior probability.

This raises serious questions for null-hypothesis statistical-significance testing, as it implies the null hypothesis of 0 will always be rejected with sufficient data, meaning that a failure to reject only implies insufficient data, and provides no actual test or confirmation of a theory. Even a directional prediction is minimally confirmatory since there is a 50% chance of picking the right direction at random.

It also has implications for conceptualizations of theories & causal models, interpretations of structural models, and other statistical principles such as the “sparsity principle”.

https://gwern.net/doc/genetics/heritable/correlation/2007-smith-figure1-correlationdistribution.jpg

Knowing one variable tells you (a little) about everything else. In statistics & psychology folklore, this idea circulates under many names: “everything is correlated”, “everything is related to everything else”, “crud factor”, “the null hypothesis is always false”, “coefficients are never zero”, “ambient correlational noise”, Thorndike’s dictum (“in human nature good traits go together”1), etc. Closely related are the “bet on sparsity principle”2, Anna Karenina principle, Barry Commoner’s “first law of ecology” (“Everything is connected to everything else”) & Waldo R. Tobler’s “first law of geography” (“everything is related to everything else, but near things are more related than distant things”).3

The core idea here is that in any real-world dataset, it is exceptionally unlikely that any particular relationship will be exactly 0 for reasons of arithmetic (eg. it may be impossible for a binary variable to be an equal percentage in 2 unbalanced groups); prior probability (0 is only one number out of the infinite reals); and because real-world properties & traits are linked by a myriad of causal networks, dynamics, & latent variables (eg. the genetic correlations⁠ which affect all human traits, see ⁠heat maps in appendix⁠ for visualizations) which mutually affect each other which will produce genuine correlations between apparently-independent variables, and these correlations may be of surprisingly large & important size....

....MUCH MORE
*
The first law of ecology: "Everything is connected."

And Mr. Burke? 

November 2013
.....The law made a small fortune for James Burke:
Connections explores an Alternative View of Change (the subtitle of the series) that rejects the conventional linear and teleological view of historical progress. Burke contends that one cannot consider the development of any particular piece of the modern world in isolation.

Rather, the entire gestalt of the modern world is the result of a web of interconnected events, each one consisting of a person or group acting for reasons of their own (e.g., profit, curiosity, religious) motivations with no concept of the final, modern result of what either their or their contemporaries' actions finally led to. The interplay of the results of these isolated events is what drives history and innovation, and is also the main focus of the series and its sequels.

To demonstrate this view, Burke begins each episode with a particular event or innovation in the past (usually Ancient or Medieval times) and traces the path from that event through a series of seemingly unrelated connections to a fundamental and essential aspect of the modern world. For example, The Long Chain episode traces the invention of plastics from the development of the fluyt, a type of Dutch cargo ship.
Watch the full documentary now (30 episodes, each 45 minutes long)
Connections (1978)
1. The Trigger Effect details the world’s present dependence on complex technological networks through a detailed narrative of New York City and the power blackout of 1965.

2. Death in the Morning examines the standardization of precious metal with the touchstone in the ancient world.

3. Distant Voices suggests that telecommunications exist because Normans had stirrups for horse riding which in turn led them to further advancements in warfare.

4. Faith in Numbers examines the transition from the Middle Ages to the Renaissance from the perspective of how commercialism, climate change and the Black Death influenced cultural development.

5. The Wheel of Fortune traces astrological knowledge in ancient Greek manuscripts from Baghdad’s founder, Caliph Al-Mansur, via the Muslim monastery/medical school at Gundeshapur, to the medieval Church’s need for alarm clocks (the water horologium and the verge and foliot clock).

6. Thunder in the Skies implicates the Little Ice Age (ca. 1250-1300 AD) in the invention of the chimney, as well as knitting, buttons, wainscoting, wall tapestries, wall plastering, glass windows, and the practice of privacy for sleeping and sex.

7. The Long Chain traces the invention of the Fluyt freighter in Holland in the 1500s. Voyages were insured by Edward Lloyd (Lloyd's of London) if the ships hulls were covered in pitch and tar which came from the colonies until the American Revolution in 1776.

8. Eat, Drink and Be Merry begins with plastic, the plastic credit card and the concept of credit then leaps back in time to to the Dukes of Burgundy, which was the first state to use credit
.
...MUCH MORE, although the original links to YouTube have been pulled, a bit of searching (cough*vimeo*cough) shows the vids are still on the web.

"James Burke’s New Project Aims to Help us Deal with Change, Think Connectively, and Benefit from Surprise"