Another repost, this one from November 2023.
I've mentioned that during a misspent youth I, among other things, pretty much memorized the Cowles Commission's Common Stock Indexes 1871-1937. The Cowles gang moved from Colorado Springs to the University of Chicago and thence to Yale University where, after changing its name to The Cowles Foundation for Research in Economics, it operates under the watchful eye of Professor Robert Shiller.
In addition to Shiller some of the other Cowles worker bees, Tjalling Koopmans, Kenneth Arrow, Gerard Debreu, James Tobin, Franco Modigliani, Herbert Simon, Lawrence Klein, Trygve Haavelmo and Harry Markowitz, all went on to win Nobel econ prizes.
For years I have mistrusted any statistical interpretation of equity returns that went back any further than Cowles 1871 start date because, as he said in the monograph (a freakin' long monograph, 506 pages) as he said, the gaps in the data on pricing and on dividends as well the selection bias, too many banks, not enough industrial and commercial equities, as well as survivorship bias—eliminate the canals and wooden highway companies and what remains gives the impression of better returns than were actually experienced by portfolio investors.
My view hasn't really changed as new data is slowly uncovered, the problems of, say, reconstructing the dividend history of New York Guano during the 1850's - 60's remains.
Here's a post from 2009 with some of that dubiosity: "Does Stock-Market Data Really Go Back 200 Years?" and another from 2011 Research Affiliates (Rob Arnott) Fundamentals Newsletter: "THE BIGGEST URBAN LEGEND IN FINANCE".
So, for readers who have been with us for a while this stuff is old hat. For folks new to the blog and for readers desiring to be as au couarnt as possible (this paper is six days old!) here we go.
From the Financial Analysts Journal (Taylor and Francis) [which Arnott used to edit]:
Received 13 Feb 2023, Accepted 05 Oct 2023, Published online: 13 Nov 2023
Abstract
When Jeremy Siegel published his “Stocks for the Long Run” thesis, little was known about 19th-century stock and bond returns. Digital archives have made it possible to compute real total return on US stock and bond indexes from 1792. The new historical record shows that over multi-decade periods, sometimes stocks outperformed bonds, sometimes bonds outperformed stocks and sometimes they performed about the same. New international data confirm this pattern. Asset returns in the US in the 20th century do not generalize. Regimes of asset outperformance come and go; sometimes there is an equity premium, sometimes not.Must stocks beat bonds over the long run? Should investors expect a stock portfolio to compound in real terms at 6% to 7% per year?
Over the last three decades, Jeremy Siegel has defended both claims under the rubric of “Stocks for the Long Run.” He built on foundations laid by Ibbotson and Sinquefield (Citation1976), who showed substantial outperformance for stocks over bonds dating back to 1926, and on Shiller (Citation2015), who found strong stock performance dating back to 1871. By the 1980s, the degree to which stocks had outperformed bonds, as shown in the historical record, had begun to puzzle theorists (Mehra and Prescott Citation1985).
Siegel’s contribution was to push the beginning of the stock record back from 1871 to 1802, drawing on the much earlier data collection efforts of Smith and Cole (Citation1935),Footnote1 and to push the bond record back from 1925 to 1802, drawing on the compilation in Homer (Citation1963). In later book editions, Siegel (Citation2014) was able to update portions of the 19th-century stock record using data from Goetzmann, Ibbotson, and Peng (Citation2001), without disturbing the thesis of a persistent, strong equity premium.
However, in the 30 years since Siegel (Citation1992a, Citation1992b) first advanced his thesis, new data have emerged on 19th-century stock and bond returns. The new findings substantially diverge from the record available to Siegel and are at variance with the 20th-century record in the annual compilation maintained by Ibbotson (Citation2020) in the Stocks, Bonds, Bills and Inflation yearbook. This paper provides a summary of the new findings and considers the implications for investment practice.Footnote2 The post-1926 period in the US emerges as highly distinct within a broader historical view, with outcomes that do not generalize to other times and places.
The New Historical Record
The online Appendix links to working papers which detail the methods used to collect the new data. The Appendix also links to files containing the raw security-level data, hosted by Financial Analysts Journal for use by future researchers and to allow auditing of the findings.
Prices
A team led by Richard Sylla examined hundreds of newspapers published before 1860 and compiled prices on a much larger set of securities than had previously been available (Sylla, Wilson, and Wright Citation2006). After 1860, I hand-gathered prices from the American Railroad Journal, Banker’s Magazine, the Commercial and Financial Chronicle (CFC), and other contemporary publications, through 1897 (stocks) and 1926 (bonds), supplemented by the collections of Hall, Payne, and Sargent (Citation2018), Macaulay (Citation1938), and Martin (Citation1898). The raw price data were condensed into an annual series of average January prices, as Cowles (Citation1939) had done, the better to link to his post-1871 series (Shiller Citation2015).Share CountsThe new stock and bond series both begin in January 1793, the first month where more than three stocks were found trading regularly.Footnote3 The new stock series ends January 1897; returns did not change much when survivorship bias in the Cowles (Citation1939) series from 1871 to 1897 was corrected (McQuarrie Citation2020). Accordingly, following 1896 the stock series and following 1925 the bond series used by Siegel are appended to complete the record.Footnote4
Share counts were sought to get capitalization-weighted stock returns. The development of digital archives of magazines and newspapers and the digitization of out-of-copyright books made this possible. An example is Goddard (Citation1831), which contains share counts for most of the larger stocks of that era. Corporate biographies and state legislative reports were also accessed, along with early compilers such as Poor (Citation1860).Footnote5
Dividends and Distributions
Information on dividends was gathered to construct an index of the total return on stocks. Siegel’s initial sources did not observe dividends; he had to estimate dividend yields. Later, when Goetzmann, Ibbotson, and Peng (Citation2001) became available, Siegel used their post-1824 data on dividends. The Goetzmann et al. data were limited to stocks in New York and, as data collection proceeded, were found to contain numerous omissions but also unique information, which supplemented the efforts described next.The same digitized sources tapped for share counts also provided many dividends. Nonetheless, before 1865 the record of semi-annual dividends remained spotty. I then discovered that early firms would publish a notice in the local newspaper when a dividend was declared. Digitized records were searched for the term “dividend” year by year, city by city. That search, supplemented by the Goetzmann et al. data, provided what appeared to be 95% or better coverage of the dividends paid by stocks in the new database.
After 1865, the CFC, annual Poor’s editions, and Martin provided the dividend record. These sources were also used to track share splits, stock dividends, and rights issues.
Summary
The new record improves on the old as follows.
Includes securities trading outside of New York, in Boston, Philadelphia, Baltimore, and southern and western cities. The new record covers three to five times more stocks and five to ten times more bonds.
The expanded coverage captures more failures, reducing survivorship bias. Outside of New York, banks failed, turnpikes succumbed to canals, canals lost to railroads, and new railroads fell on hard times and never paid a dividend. States defaulted in the Panic of 1837. Corporate bonds were downgraded or defaulted.
Includes federal, municipal, and corporate bonds, and large numbers of each, as opposed to the one bond used each year by Siegel prior to 1862. The new bond record observes price changes, where Siegel had to infer price change from successive yields.
Calculates capitalization-weighted total return for stocks. The old stock record was either price-weighted (Goetzmann, Ibbotson, and Peng Citation2001) or equal-weighted (Smith and Cole Citation1935) and lacked information on dividends.
Shortcomings remain. These include: (1) annual frequency; (2) time-averaged data (Schwert Citation1990); (3) exclusion of stocks that traded over the counterFootnote6; and (4) inability to observe ex-dividend and ex-coupon dates and prices.
Key ChangesCorrected History of US Stock and Bond Performance
Stock returns before 1871 look different now because of the reduction in survivorship bias. The old record omitted the largest stock that traded before the Panic of 1837, the 2nd Bank of the United States. At the peak before the Panic hit, the 2nd BUS accounted for almost 30% of total US market capitalization. It failed as the Panic proceeded, with shares dropping in price from $120 to $1.50, and never recovered.Bond returns look more positive now, due to inclusion of corporate bonds, inclusion of a broader selection of federal and municipal bonds, and an adjustment for the greenback price of interest paid in gold coin between 1862 and 1879.Footnote7
...shows the new record of US stock and bond performance from 1792. It can be compared to Figure 5-4 in Siegel (Citation2014, 82). Two recent periods are marked out in, with the bond performance line reset at the beginning of the right panel to facilitate comparison.Figure 1. Stock and Bond Performance from 1792
Note. Performance through December 2019 (pre-pandemic). The bond performance line (dark brown) is reset equal to stock wealth at the end of 1981 to facilitate comparison in the years that follow.
