The Singularity is closer and dumber than you think.
Forget Skynet. The real danger isn't superintelligent machines. It's powerful dumb ones.
Three items combined serendipitously to bring this peril to my attention: William Hertling's sci-fi novel Avogadro Corp., a New Scientist article about an app that generates emails that fake empathy, and a New York Times blog post on the dangers of artificial stupidity. As the line of the cover of Hertling's novel warns, "The Singularity is closer than it appears." Closer and klutzier.
In Hertling's 2014 novel, computer genius David Ryan heads the Email Language Optimization Project at Avogadro Corporation (a very thinly disguised stand-in for Google), where his team has created ELOPe—an app that helps users "craft more compelling, effective communications." In order to persuade, ELOPe reads through the emails received and sent by the target. Based on what it finds, the app makes suggestions for word choices, data, reasoning, and emotional appeals that will motivate the recipient to act as the sender wants.
Ryan describes his new app as the biggest improvement to email since spell-check and grammar check. Unfortunately, a hostile Avogadro Corp ops manager wants to kill the ELOPe project because he thinks that it is using too many of the company's computational resources. In desperation, Ryan modifies ELOPe, instilling it with the goal of doing whatever it must to persuade people to grant it the resources it needs. A flood of emails ensue, and let's just say the world becomes a pretty interesting place thereafter.
Shortly after finishing Avogadro Corp, I came across New Scientist's article about the Crystal Knows app. Crystal Knows, which bills itself as the "biggest improvement to email since spell-check," promises that it can "show you the best way to communicate with any coworker, prospect, or customer based on their unique personality." How? By applying its algorithm to the online information about a recipient and then helping you to select "the words, phrases, style, and tone you should use to reach the recipient in the way that they like to communicate, rather than your own." Instant empathy.
Crystal Knows is far from alone in trying to figure out how to push your empathy buttons. For example, Persado is an automated persuasion platform with a personality analysis algorithm; it generates marketing language and emotional insights for client companies aiming to motivate their customers and stakeholders.
The third serendipitous bit of reportage the provoked my interest was a blog post, "The Real Threat Posed by Powerful Computers," by New York Times technology reporter Quentin Hardy. "If the human race is at peril from killer robots," he argues, "the problem is probably not artificial intelligence. It is more likely to be artificial stupidity." Basically, Hardy thinks the threat comes from programs and machines that are over-optimized to achieve a task.
In his 2014 book Superintelligence: Paths, Dangers, Strategies, the Oxford philosopher Nick Bostrom outlined a scenario in which a very powerful computer is programmed to make paper clips. The machine brilliantly and relentlessly pursues this goal and prevents anyone from attempting to change its paper clip imperative. Eventually, the Earth is a mass of paper clips and the computer sets its sights on the rest of the universe....MORE
