From The Economist, July 2:
We used artificial intelligence to test the accuracy of our forecasts
The tone of pronouncements made in the leader pages of The Economist has been likened to the “voice of God”. The comparison is not meant flatteringly. But it raises a fair question. A deity would be omniscient. How accurate, by comparison, are our predictions? Rigorous fact-checking usually saves us from embarrassing errors about the present. The future is trickier.
In 1999 we said oil could fall to $5 a barrel, from around $10 at the time. That $10 turned out to be a generational low. Prices rose more than tenfold in the next decade. Then, in 2013, we called a peak in global oil demand. Today it is still stubbornly trundling upwards. This April we said that oil markets were in “La La land” about the war in Iran and predicted a spike in prices. They have since fallen by about a third.Perhaps the black stuff is our bête noire. Yet these howlers, and others, have given rise to the charge that, far from being all-knowing, The Economist is so reliably wrong that the smart move is to bet against it. There is no better time to double down on a stock-market rally than when we start fretting about a bubble. Last year the then chairman of Reform UK, a populist-right party, called The Economist the “ultimate contrarian indicator”, while complaining about our criticism of his party’s fantasyland fiscal numbers. (A few months later, the party reversed course and ditched those policies.)Is the accusation fair? Or can The Economist lay claim to a creditable forecasting record? Predictive perfection is not our goal and we aim mainly to inform and stretch minds. But ideally we would be right more often than wrong. To assess our record with something approaching neutrality, we took the 7,000 or so leaders The Economist has published this millennium and fed them into GPT-5.5, an artificial-intelligence model.We asked it to assess whether each leader had made a falsifiable claim about the future as part of its main thesis. About 1,400 did. We then extracted those predictions, and asked the AI to mark out of ten both how contrarian the leader’s outlook was at the time and how accurate the prediction turned out to be. We ran those queries several times and took an average.Overall, the AI assessor’s conclusions are reassuring, at least for those of us who make a living writing (and occasionally predicting) for The Economist....*****...To evaluate our record in detail, start by casting your mind back to the turn of the millennium, when our leader sample begins. We were preoccupied by transformative technology. Market optimism seemed to be teetering on the delusional. As the dotcom crash took hold, we fretted (accurately) about America’s economic pain spreading to Europe and Asia, and (unnecessarily) about a second dip in growth. But soon we began to fulminate about the next crisis—which started in America’s housing market and culminated in the worldwide financial mayhem of 2007-09....
....MUCH MORE
