Thursday, November 7, 2013

When Writing Meets Datamining or Rise of the Journo-Analyst

From Fast Company:

How Vocativ Mines The "Deep Web" For Storytelling 
The startup, funded by security tech magnate Mati Kochavi, adapts technology used by hedge funds and intelligence agencies to find news all over the world.

Back in 2012, a group of digital journalists went hunting for Ugandan warlord Joseph Kony. They tried to track him using a trove of data--like mercenary chatter found on an obscure corner of the web. In the end, they weren't exactly able to string together enough information to triangulate his position. But Kony wasn't the only signal they were tracking.

By setting geographic parameters for a data-analysis operation and opening their ears, the analysts and journalists of a new kind of news organization called Vocativ stumbled upon talk on a message board about something equally as curious as the Lord's Resistance Army's leader's movements. They found a thriving Facebook subculture of gun sweepstakes where gun shops and industry publications would give away weapons, including AR-15 rifles, to random fans who “liked” their page.

The Facebook guns story wasn't what reporters and analysts went looking for. And it was only fleshed out by mining the unindexed, un-Google-able Web. Monitoring of deep Internet chatter from Syria has also led Vocativ to stories on sex tapes of prominent Syrians being used as propaganda by both sides in that country's bloody civil war.

Vocativ's CEO Scott Cohen (formerly digital executive editor of The New York Daily News) notes that the Web is full of what he calls “clusters of disparate signals”--and that his organization's job is to organize them into coherent stories. A big part of this is following horizontal threads of data--the type of leads hedge funds, law enforcement, and intelligence agencies thrive on--with the hope of unearthing unexpected information that's of interest to the public. As Vocativ's application trawls the Web for interesting potential news stories, analysts and their journalist cohorts verify sources' accuracy (often with non-English speakers). Then the journalist-analyst pairs translate the raw materials into stories designed to challenge Vice and the Wall Street Journal, among others. The pairing of often young and uniquely experienced journalists with data analysts to produce actual stories (not just files of raw data leads) is what makes Vocativ's business unique.

Agnostic, relatively unbiased search parameters to monitor the web for hidden news is the big idea behind Vocativ, which launches today. (Vocativ has been in not-so-stealth mode, with a different site design, for much of the year.) Employees of the digital news agency come from Vice, Huffington Post, ABC, The New York Daily News, and more, and they speak a wide variety of languages. Vocativ is based in New York with outposts around the world. One of its big goals is to use the deep web as a primary source.

The “deep web” consists of all the things available on the Internet that standard search engines overlook--things like spreadsheets and Word documents, subscription-only journals and pages with dynamic content. Vocativ's principals claim they can use the deep web, combined with monitoring of social media in a host of foreign languages, to find news stories other agencies can't. Their search technology is similar to that used by law enforcement to detect terrorist chatter, hedge funds to find hidden financial information, and by intelligence agencies to gauge sentiment and collect intelligence....MUCH MORE