Thursday, June 14, 2012

"Using Artificial Intelligence to Take the Market’s Pulse"

If you don't have secret Greek polling data...
From Institutional Investor:

Harnessing the power of language can be a boon to traders bent on something as seemingly subjective as a diagnosis of the market’s pulse. It’s one thing to scour news databases to find negative references to a company whose stock you’re about to sell short. But it’s another thing to detect what Philip Resnik, a professor of linguistics at the University of Maryland, calls spin.

“Language is ambiguous — words have multiple meanings,” says Resnik, who’s also a member of the university’s Institute for Advanced Computer Studies.

Traders have been mining for data in news archives for decades. And they’ve long known how to detect the mood of the masses by assessing puts and calls on stock options. But today, computers armed with the latest artificial intelligence, or AI, software are learning to decipher how the public uses language and which combinations of words can predict the kind of phenomena that interest investors and pollsters.

For example, when a professor at Carnegie Mellon University searched for negative words in news databases — “unemployment,” “layoffs” and “fear” — he noticed a high correlation with a drop in the U.S. Consumer Confidence Index. His project worked just fine until the fall of 2010, when he deduced from the frequency of certain positive words, including “jobs” and “employment,” that public sentiment was about to rise.

The problem: Apple had just released a new iPhone; all those “jobs” references were mentions of company co-founder Steve Jobs. Consumer confidence was in fact falling, but news reports on the hip new Apple product made the scientist erroneously predict that consumer confidence would soon shoot through the roof.

Or think about it this way: A person describing a movie as “a bomb” probably didn’t like the flick. But if he says the movie was “the bomb,” he’s using slang to say how much he enjoyed it. An AI-enabled computer can distinguish between the two uses of the word. And there’s another difference in how traders today are trolling for sentiment: their data sources. Twitter alone has 140 million users churning out thoughts on products, companies and current events around the clock.

Back in 2005, Paul Tetlock, now a professor at Columbia Business School in New York, wrote an academic paper when he was at the University of Texas demonstrating how content in the business press predicts stock market movements. He found not only that high levels of what he calls media pessimism predict downward swings in the stock market, but also that investors can use media pessimism to predict trading volume....MORE
In a case of the news not being compliant with the story the Financial Times had this article on May 24:
Last tweet for Derwent’s Absolute Return
The only dedicated “Twitter” hedge fund has shut down after deciding to offer its social media indicators to day traders instead.

Derwent Capital Markets’ Absolute Return fund was quietly liquidated just a month after starting up last year. Paul Hawtin, chief executive and founder of Derwent Capital, said that one of its largest investors suggested taking the trading signals to private investors rather than trading on them directly.

“As a result we made the strategic decision to close the Derwent Absolute Return fund and invest directly in developing an online trading platform,” he said.

During the one month for which the fund was open, it returned 1.86 per cent in its sterling shares, ahead of the market and the average hedge fund.

Derwent examines tweets from users of Twitter, as well as messages posted on Facebook and other social media, and creates measures of sentiment towards individual stocks, wider markets and even commodities. Mr Hawtin said that moves in sentiment tended to give a lead of about three days on price moves....MORE