Friday, September 22, 2023

The Lindy Effect: "for many kinds of entity: the longer they have been around so far, the longer they are likely to last."

 Via arXiv.org:

The Lindy Effect 
Toby Ord (University of Oxford) 

The Lindy effect is a statistical tendency for things with longer pasts behind them to have longer futures ahead. It has been experimentally confirmed to apply to some categories, but not others, raising questions about when it is applicable and why. I shed some light on these questions by examining the mathematical properties required for the effect and generating mechanisms that can produce them. While the Lindy effect is often thought to require a declining hazard rate, I show that it arises very naturally even in cases with constant (or increasing) hazard rates — so long as there is a probability distribution over the size of that rate. One implication is that even things which are becoming less robust over time can display the Lindy effect.

One book has been in print for three years; another for three hundred. Which should we expect to go out of print first?

The Lindy effect is a statistical regularity where for many kinds of entity: the longer they have been around so far, the longer they are likely to last. This was first clearly posed by BenoƮt Mandelbrot (1982, p. 342) in his book, The Fractal Geometry of Nature:

‘However long a person’s past collected works, it will on the average continue for an equal additional amount. When it eventually stops, it breaks off at precisely half of its promise.’
Mandelbrot called this effect ‘Lindy’s Law’ in honour of an anecdote by Albert Goldman (1964) about how the future career length of a comedian might be predicted from their past exposure.1

[1 It is not clear whether Goldman, or the denizens of Lindy’s Deli he describes, were really describing the same principle as Mandelbrot.]

The idea was developed by Nassim Taleb (2012) in his book, Antifragile. The book focused on a special category of entities: those which aren’t weakened by exposure to shocks and stresses, but which instead become stronger and more robust. He describes the Lindy effect in those terms (p. 318): 

‘If a book has been in print for forty years, I can expect it to be in print for another forty years. But, and that is the main difference, if it survives another decade, then it will be expected to be in print another fifty years. This, simply, as a rule, tells you why things that have been around for a long time are not “aging” like persons, but “aging” in reverse. Every year that passes without extinction doubles the additional life expectancy. This is an indicator of some robustness. The robustness of an item is proportional to its life!’

Taleb (pp. 317–18) suggests that this Lindy effect applies to all things without a natural hard upper bound to their lifespans (those that are ‘non-perishable’2):

‘For the perishable, every additional day in its life translates into a shorter additional life expectancy. For the nonperishable, every additional day may imply a longer life expectancy.’

The Lindy effect is an important and much-discussed principle but has had surprisingly little formal development. This paper attempts to address that need. In doing so, it takes a synoptic view: connecting the Lindy effect to results and techniques from statistics, reliability analysis, economics, population ecology, and cosmology.

Which distributions produce which Lindy effect?....

....MUCH MORE