"Artificial intelligence: machine v man"

From FT Magazine:

Computers will soon become more intelligent than us. Some of the best brains in Silicon Valley are now trying to work out what happens next

The scene in the cramped office in Berkeley on a recent Saturday feels like a typical start-up carried along by the tech boom, with engineers working through the weekend in a race against time. The long whiteboard down one wall has been scrawled over in different-coloured pens. A large jar of candy and a glass-doored fridge full of soda sit by the entrance.

Nate Soares, a former Google engineer, is sitting on the edge of a sofa weighing up the chances of success for the project he is working on. He puts them at only about 5 per cent. But the odds he is calculating aren’t for some new smartphone app. Instead, Soares is talking about something much more arresting: whether programmers like him will be able to save mankind from extinction at the hands of its own most powerful

The object of concern – both for him and the Machine Intelligence Research Institute (Miri), whose offices these are – is artificial intelligence. Super-smart machines with malicious intent are a staple of science fiction, from the soft-spoken Hal 9000 to the scarily violent Skynet. But the AI that people like Soares believe is coming mankind’s way, very probably before the end of this century, would be much worse.

If it were a sci-fi movie, a small band of misfits would be thrown together at this point to save the planet. To the people involved in this race, that doesn’t seem so far from reality. Besides Soares, there are probably only four computer scientists in the world currently working on how to programme the super-smart machines of the not-too-distant future to make sure AI remains “friendly”, says Luke Muehlhauser, Miri’s director.

Their effort is prompted by a fear of what will happen when computers match humans in intelligence. At that point, humans would cede leadership in technological development, since the machines would be capable of improving their own designs by themselves. And with the accelerating pace of technological change, it wouldn’t be long before the capabilities – and goals – of the computers would far surpass human understanding.

In their single-mindedness, they would view their biological creators as mere collections of matter, waiting to be reprocessed into something they find more useful, says Muehlhauser. They would consume all the resources on earth before propelling themselves into space, sucking energy from distant stars and ultimately devouring much of the visible universe....MUCH MORE