The current, accepted wisdom is that those who understand Big Data – the enormous datasets of information being collected with nearly every click of every computing device on the planet – will rule the roost in the future. Presumably, if you can predict behavior by measuring and monitoring people’s machines down to an almost atomic level, you can make both your customers and your shareholders much happier. (See Google.)
There’s just one problem: outside of companies like Google that have long made use of rich rosters of PhDs, there are nowhere near enough “data scientists” — graduate-level candidates with backgrounds in machine learning or statistics — to analyze the massive streams of information that are being produced, and that gap is growing by the day.
Cofounder Ali Behnam of Riviera Partners, a Palo Alto-based technology recruiting firm, says he’s aware of “thousands” of data science jobs that are awaiting candidates at nearly every type of Bay Area-based tech company. (Even Riviera Partners employs a couple.)
And there are exponentially more nationally, according to McKinsey & Co., which estimates that the U.S. has roughly 140,000 to 190,000 fewer people with analytic expertise than it needs, and that things are going to grow worse. To wit, McKinsey projects that by 2018, the U.S. will need 60 percent more people with advanced degrees in statistics of machine learning than will be available.
“Not a lot of engineers grow up saying, ‘Gee, I want to be a data scientist,’” notes Teri McFadden, a recruiting VP at Norwest Venture Partners. She says that there are “nowhere near” enough candidates with data science backgrounds to fill the openings she sees at Norwest’s portfolio companies.
Solutions to the situation are far from clear-cut. While some startups with deep pockets can pay top dollar now, the rates are rising so fast that even they may not be able to keep up.The link came via this morning's peHUB post on some folks addressing the shortages:
Behnam says pay for data scientists has rocketed from $125,000 to $150,000 two years ago to upwards of $225,000 these days – even for those straight out of school. In cases where a data scientist has a few years of experience in the working world, pay can reach even more dizzying levels. Stephen Purpura, the co-founder and CEO of Seattle-based software company Context Relevant, recently lost out on one job candidate who had with a PhD and seven years of work experience. He was dying to land the guy. As he puts it, “These people are almost like unicorns.” But out of the blue, Microsoft came knocking with an offer of $650,000 in annual salary and guaranteed bonuses. “We can’t compete with that kind of offer,” says Purpura.
For those hoping that equity can bridge the gap, it can’t, say those doing the wooing. “A lot of these folks don’t have that entrepreneurial, I’m-going-to-make-a-bazillion-dollars-at-a-startup type mentality,” observes McFadden. “Their motivations aren’t the same as many software engineers.” It’s something that venture capitalist Venky Ganesan of Globespan Capital Partners has seen repeatedly in his firm’s attempts to help its portfolio companies. (He said two data scientists recently joined two of Globespan’s portfolio companies, leaving a whopping 18 slots to fill.)...MORE
Ayasdi Raises $30M to Turn Employees into De Facto Data Scientists
