From the sadly defunct Kernel magazine, December 2015:
It’s safe to say that most of us probably don’t spend a lot of time thinking about algorithms. We go to Amazon and notice we have new recommendations based on our previous purchases; we see that Netflix has carefully selected movies based on our past preferences. But we don’t necessarily think about what that means, or what’s going on behind the scenes. We probably think most about algorithms when they go awry, whether that’s yet another Facebook faux pas, a stomach-dropping flash crash on Wall Street, or a online shop selling tasteless T-shirts generated entirely by computers.
It’s in those moments that we’re reminded just how much of the world runs on algorithms: the sets of rules, increasingly byzantine and incomprehensible to humans, that govern all the computers around us. We’re reminded just how vulnerable we are when algorithms go bad (obligatory Skynet reference here); we’re reminded that their mistakes are not those humans make because, of course, algorithms are not human.
Pedro Domingos has spent a lot of time thinking about algorithms. His new book, The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World, is an introduction to that world—and a report on the state of the art. He believes that we live in an age of algorithms, and sees a time when we may see a time when they radically remake our world—even more than they already have.
Via email, we discussed the difference between human and computer “thinking” and learning, how much of our lives are already influenced by algorithms, and what happens when the machines finally learn how to learn everything.Domingos is pretty important to a lot of the companies attempting to exploit Artificial Intelligence:
You say we live in the age of algorithms. How so, and why might we be unaware of just how many algorithms are working for—and possibly against—us every day? How is machine learning working behind the scenes in ways we don’t necessarily realize?
Everything computers do, they do using algorithms. Your cellphone, your laptop, your car, your house, and your appliances are all full of algorithms. But they’re hidden: You see the shiny gadget, but not what’s going on inside. Siri uses an algorithm to understand what you say, Yelp uses an algorithm to select restaurants for you, your car’s GPS system uses an algorithm to find the best route there, and the credit card reader uses an algorithm to take your payment. Companies use algorithms to select job applicants, mutual funds use them to trade stocks, and the NSA uses them to flag suspicious phone calls.
The difference between “regular” algorithms and learning algorithms is that the former have to be manually programmed by software engineers, explaining step by step what the computer needs to do, while the latter figure it out on their own by looking at data: Here’s the input, here’s the desired output, how do I turn one into the other? And what’s remarkable is that the same learning algorithm can learn to do an unlimited number of different things—from playing chess to medical diagnosis—just by being given the appropriate data.
What is the “master algorithm” of the title, and how is it different from, say, Ray Kurzweil’s singularity? What are some of the potential advances that the master algorithm could bring?
The master algorithm is an algorithm that is capable of learning anything from data. Give it data about the planets’ motions, inclined planes, and pendulums, and it discovers Newton’s laws. Give it DNA crystallography data and it discovers the double helix. From all the data in your smartphone, it learns to predict what you’re going to do next and how to help you. Perhaps it can even discover a cure for cancer by learning from a massive database of cancer patient records.
Other advances it could bring are: home robots; replacing the World Wide Web [with] a World Wide Brain that can answer your questions instead of just showing you Web pages; and a 360-degree recommendation system that knows you as well as your best friend and recommends not just books and movies but dates, jobs, houses, vacations—everything in your life.
Kurzweil’s singularity is the point at which artificial intelligence exceeds the human variety on Earth, and therefore becomes incomprehensible to us. Or, more precisely, that’s the “event horizon” of the singularity, like the event horizon of a black hole is the point beyond which even light cannot escape. Without the master algorithm, we won’t reach the singularity anytime soon. With it, AI will certainly accelerate, but I think we will still understand lots about the world, because the AIs will be working to serve us, by design. We may not understand how they produced what they give us, but we will understand what those products do for us, or we wouldn’t want them. Besides, the world has always been partly beyond our understanding. The difference is that now it’s partly designed by us, which is surely an improvement.
You describe the field as currently divided into “tribes,” each with a different approach to machine learning that can solve certain kinds of problems better than others, but with no tribe possessing the algorithm that can subsume all others—basically a machine-learning process that would let us answer all answerable questions. You compare that assumed master algorithm to the Standard Model of particle physics, or the central dogma of molecular biology: “a unified theory that makes sense of everything we know to date, and lays the foundation for decades or centuries of future progress.” That sounds like a big claim. What makes a master algorithm seem plausible, and what’s keeping our disconnected “tribes” from creating it?
Even some of the simplest learning algorithms have a mathematical proof that they can learn anything given enough data. So in that sense there’s no doubt that the master algorithm exists, and indeed some researchers in each of the tribes believe that they’ve already found it. But the catch is that the algorithm has to be able to learn what you want it to using realistic amounts of data and computation. Here we turn to the empirical evidence: Nature provides us with at least two instances of an algorithm that can learn anything (or almost), namely evolution and the brain. So we know that the master algorithm exists; the question is whether we can figure out what it is precisely and completely enough to write it down, in the same way that physicists write down the laws of physics as equations (which are themselves a kind of algorithm).
Unfortunately, the five tribes of machine learning are like the blind men and the elephant: One feels the trunk and thinks it’s a snake, another leans against the leg and thinks it’s a tree, yet another touches the tusk and thinks it’s a bull. What we really need is to take a step back and see the whole picture and how the pieces fit together. Ironically, this might be easier for someone who’s not already in the field and thinking along the previously laid tracks of the five tribes....MORE
August 2018
"Who needs democracy when you have data?"
April 2018
Machine Learning Guru Pedro Domingos on the Arms Race in Artificial Intelligence
February 2018
How Amazon Rebuilt Itself Around Artificial Intelligence
May 2017
"Why Google Is Suddenly Obsessed With Your Photos"
June 2016
Facebook's Race To Dominate Artificial Intelligence (FB)
