According to the just-published 2016 Rich List of the World’s Top-Earning Hedge Fund Managers by Institutional Investor’s Alpha magazine, eight of the top ten earners fall into the “quant” category, and half of the 25 richest of the year are quants. The firms listed include the likes of Renaissance Technologies, D.E. Shaw, Two Sigma, Millennium, Citadel and Schonfeld, none of which engage in “smart beta” or factor-based investments. None of these firms apply the theories published by Economics Nobel laureates such as James Tobin, Eugene Fama, Robert Shiller and dozens of others. Instead, these firms rely on a combination of mathematics and computational technology.
THE FOUNDATIONS OF FACTOR INVESTING
Consider the Capital Asset Pricing model (CAPM), the bedrock of financial economics. It was developed by William Sharpe (also a Nobel laureate) and others back in the 1960s. Almost six decades later, students around the world are still taught that the return of a security is a linear function of the risk-free rate and the “risk premium.” That’s it — your formula to riches! Forget about balance sheet information, news, sentiment, flows, frictions, … — everything can be condensed in a simple formula that can be estimated with 18th century mathematics. Since its publication in the prestigious Journal of Finance, this formula has been explained in every finance textbook, and is the foundation of the so-called “factor investing” in general and “smart beta” in particular.
Could things really be that simple? Financial markets are incredibly complex systems, where millions of individuals interact with each other exchanging information asynchronously and asymmetrically. Thousands of papers published in the Social Science Research Network (SSRN) claim to validate these theories empirically. Of course, all this evidence is based on statistical backtests, which are known to be easily manipulated.
FRAUDULENT STATISTICS
Real scientists do not validate a theory through historical simulations. They conduct experiments. Here is how it works in the experimental sciences, in this order:
However, the evidence presented in SSRN often follows a different order:
- A large number of forecasts are made (forward simulations).
- Measurements take place.
- Forecasting errors are evaluated.
- Measurements take place.
- A large number of backward simulations are made.
- Backtesting errors are evaluated, and one model is selected.
Backtesting errors are not forecasting errors, because the backtest can be modified conveniently until some algebraic expression overfits the data. This is called selection bias, which the American Statistical Association has declared to be highly misleading:
Running multiple tests on the same data set at the same stage of an analysis increases the chance of obtaining at least one invalid result. Selecting the one “significant” result from a multiplicity of parallel tests poses a grave risk of an incorrect conclusion. Failure to disclose the full extent of tests and their results in such a case would be highly misleading.
That’s right — the statistical approaches used to validate financial theories are considered inappropriate, if not fraudulent, by the main professional organization for statisticians. This has led the current President of the American Finance Association to acknowledge that most claimed research findings in financial economics are likely false. The Econometric Society is shamefully silent on this regard, ignoring the “multiple testing” problem, as if it didn’t exist. To this day, as far as we are aware, no major Econometrics textbook mentions the dangers of backtest overfitting.Hey! We linked to "Pseudo-Mathematics and Financial Charlatanism...." back in 2014.
Have you ever wondered why empirical papers in finance are rarely published in statistical journals? One reason is that most empirical finance papers would have never been published in true statistical journals, because their validation methods typically do not account for multiple testing effects. Instead, they are published in econometrics journals, most of which are not refereed by mathematicians or statisticians.
WHERE IS THE SCIENTIFIC EVIDENCE?
First, we should distinguish between academic discussion and scientific evidence. For centuries, some of the most brilliant minds in history argued philosophical and theological theories in highly formalized terms. For example, after becoming the seventeenth century’s pre-eminent scientist, Sir Isaac Newton devoted many years of his life to the pursuit of Alchemy, and to prove that the world would end in the year 2060, based on the academic analysis of the scriptures (he also predicted 2012, but alas, here we are). Newton’s academic theories about the end of the world are not rigorous, experimentally validated scientific theories, but neither are CAPM or smart beta, since they have not been subjected to rigorous, inside-out experimental analysis....MUCH MORE
And as a special bonus gift for long-suffering reader offered up:
Previously on the Mountebank channel:
UPDATED--Are You a Recent Graduate Who Hasn't Found a Job? Consider Becoming a Charlatan
Follow-up: Choosing the Charlatan Career Path
Re-post: Peak Oil Stalwart to Shutter Forum/News Site, Persue Career as Astrologer
See also:
Technical analysis
Fundamental analysis
Divination for Dummies
Pitfalls in Prognostication: Fortune Magazine's August, 2000 "Ten Stocks to Last the Decade"
