Original post:
From Institutional Investor:
There’s an algorithm for that: How big data and predictive analytics are set to transform investment research
BIG DATA IS SUDDENLY A BIG DEAL. In late May, at Michael Milken’s annual gathering of financial notables in Los Angeles, one center-stage topic was the impact of increasingly powerful computers and software and their ability to extract meaning from giant data sets. The conference featured one panel, headlined by President Obama’s 2012 campaign manager, Jim Messina, that focused exclusively on big data; Messina discussed how vast amounts of information helped secure victory. But it was a panel on trading that defined the challenge to flesh-and-blood financial and investment analysts posed by the seemingly inexorable march of algorithms, robots and big data.HT: Abnormal Returns
Louis Salkind, longtime managing director of hedge fund firm D.E. Shaw & Co., framed the challenge of big data most clearly: Salkind, who has a Ph.D. in computer science and robotics from New York University, described a mounting confrontation between analytical machines and securities analysts. Near the end of the hourlong panel, he described a world of increasing automation in which vendors create products “to shred apart Twitter and Facebook” and aggregate trading signals. “Imagine what happens when they start using big-data techniques to look at fundamental data,” he said, recounting a story about a broker who used satellite imagery of Wal-Mart Stores parking lots to forecast quarterly earnings. “When people start integrating these forms of data, it’s just going to be a different world out there.”
Salkind and D.E. Shaw have long been innovators in the use of computers for trading and investment. But some believe that the struggle between automation and human practitioners of finance, which has swept through exchanges, trading floors and even regulation and compliance, has reached a new front: securities analysis.
Computers and software clearly have advantages in tracking complex market patterns or monitoring and analyzing data points in news reports, social media and other digital sources. Computers famously never have to use the bathroom or ask for a raise, though they do break down. They are growing increasingly fast and more powerful, and they have access to far more data. Apostles of big data predict a rout of rank-and-file analysts by computers that can interpret the market with superior results. They even believe that big data will allow machines to discern the future — for an election, a stock price or a corporation — from the noise of the moment. Big data will not only reshape trading, they say, but long-term investment practices.
Others are skeptical. The future is always unpredictable, no matter how much data is aggregated and analyzed, as long as people participate in markets and economies. So far, the practical performance results have been thin: We suffer flash crashes, unhappy financial shocks and bubbles hidden in plain sight. Human analysts cannot process the flood of data that machines can, but they possess something algorithms lack: finely grained, if fallible, judgment. Human analysts can weigh murky values that may not be reducible to quantification; balance long-term and short-term perspectives; profit from intuitions about companies and their futures; forecast the evolution of technologies, brands or fads; and cope with ambiguities and complexities.
This clash of man and machine is just the most recent chapter in a centuries-old struggle that heated up when mechanical looms replaced home-based spinning wheels. Although the outcome is not clear, what is obvious is that the world of securities analysis will change under the impact of these powerful new tools. The technology may well transform the already precarious economics of securities analysis and further cull the ranks of analysts, dividing them into those who can effectively use the new techniques and those who cannot. Big data is probably here to stay. The larger question is, how do we live with it?
At the heart of this trend is the algorithm, a series of steps or instructions that tells computers how to search for and interpret data. It’s a simple but powerful concept when allied with a computer. Consumers encounter algorithms every day. In addition to routine tasks, from spell-checking to GPS route guidance to online shopping, algorithms help fly passenger jets and perform medical diagnoses, even surgeries. Soon they may drive cars.
Algorithms are also ubiquitous in finance, playing a role in everything from high frequency trading to complex valuation calculations to economic forecasting. They feed off information that washes over global finance daily — data now measured in petabytes, or billions of megabytes. No army of humans could outprocess these algorithms in weeks, much less in the fractions of a second they need to churn through data....MUCH MORE
