"How a Lone Hacker Shredded the Myth of Crowdsourcing"

A quick search of the blog shows a distinct lack of interest in the concept:

http://climateerinvest.blogspot.com/search?q=crowdsourcing

And oddly enough not may other folks care about the idea anymore either.

From Medium:

High-tech analysis of a 2011 DARPA Challenge shows why we can’t have nice things

Meet Adam. He’s a mid-level engineer at a mid-level software company in a cookie-cutter California office park. He can code a handful of languages, has a penchant for computer vision and enjoys soccer and skiing. In short, Adam has little to distinguish him from legions of other programmers in the Bay Area. Except that over a couple of nights in 2011, he stopped thousands of people from sharing in $50,000, nudged the American military in a new direction, and may have changed crowdsourcing forever.

This is the previously untold story of how and why Adam humbled some of the brightest brains in computer science, their years-long search to find him, and the researchers who now believe that the wisdom of the crowd might be nothing more than a seductive illusion.

To understand why Adam was able to make such an unwelcome impact, we must go back to 2009. Back then, the ability of crowdsourcing to crack big problems seemed unlimited. That was in large part due to DARPA’s Network Challenge, a competition organized by the Pentagon’s R&D agency to locate large balloons hidden in plain sight across the United States.

The task, declared “impossible” by one senior intelligence analyst, was actually solved in a matter of hours by a team of MIT students and scientists with the help of crowdsourcing and social networks. They developed a recursive incentive scheme that split the Challenge’s $40,000 prize money between the finders of each balloon, their recruiters, the people who recruited them, and so on. Over five thousand people joined MIT’s pyramid scheme, which DARPA later called “a clear demonstration of the efficacy of crowdsourcing.”

The high profile challenge spawned further contests, including DARPA’s follow-up Shredder Challenge, in 2011. The Shredder Challenge had clearer intelligence applications. Participants had to piece together documents sliced and diced using high-end shredding machines — the kind of evidence that military operatives might find at terrorist training camps. Five handwritten documents were shredded into thousands of tiny pieces half a centimeter long. The first puzzle had just a couple hundred shreds, the fiendish final one over 6,000. Images of these minuscule chads were posted online, and the first team to reconstruct the pages would win $50,000.

“If finding the balloons was a sprint, the Shredder Challenge was a marathon,” remembers Manuel Cebrian, who was part of MIT’s winning team in the earlier challenge and ready for a new adventure. “We had to get a crowd really, really engaged for weeks rather than hours.”

It would be the perfect test for Cebrian. The energetic computational social scientist divides his time between the University of Melbourne in Australia and MIT in Cambridge, his research focusing on how social networking can make it easier to find people and tackle real-world problems like global epidemics and disaster response.

For the Shredder Challenge, Cebrian turned to new collaborators: smart grad students at the University of California San Diego (UCSD) who wanted to repeat Cebrian’s success at MIT. They were researchers in crypto-analysis, game theory and network science. “My role was to be very enthusiastic and then get these people to do the difficult work,” Cebrian says with a laugh. The team swiftly settled on rewards similar to those in the Network Challenge. If it won, users would get $1 for each edge they matched correctly. The person who recruited them would receive 50 cents, and the person above that a quarter. Though the UCSD group was not the only one to use crowdsourcing, it was the only competitor planning a completely open platform, allowing anyone, anywhere, to join online.

But this time Cebrian would not be competing just against other crowds. Some of the 9,000 teams that signed up were using sophisticated algorithms to automatically match the myriad pieces with machine learning and computer vision. (Despite this being the world’s toughest jigsaw puzzle, a few people even tried to solve the puzzles manually. Tellingly, only 70 teams managed to complete even the easiest of the five puzzles).

To make matters worse, Cebrian’s group didn’t get going until two weeks after the competition started on October 27. They quickly developed a web interface and collaborative work space for the crowd to re-assemble the documents — essentially a giant virtual jigsaw mat. But they didn’t have time to construct digital defenses, such as verifying users’ identities or limiting their access to completed sections of the puzzle. “We were crossing our fingers, hoping we wouldn’t get sabotaged,” says Wilson Lian, the team’s security expert.

Trouble in the crowd

At first the hive mind functioned flawlessly. Cebrian’s winning history helped recruit over 3,600 users, who blasted effortlessly through the simpler puzzles. Individual players made errors, of course, but nearly 90 percent of those mistakes were fixed by others in the crowd in a matter of minutes. In just four days, the UCSD group had rebuilt the first three documents and was rated second overall. In late November DARPA updated its leader board to reflect UCSD’s meteoric progress — and that’s when their troubles began....MUCH MORE