Welcome to the Next Level of Bullshit
One of the most salient features of our culture is that there is so much bullshit.” These are the opening words of the short book On Bullshit, written by the philosopher Harry Frankfurt. Fifteen years after the publication of this surprise bestseller, the rapid progress of research on artificial intelligence is forcing us to reconsider our conception of bullshit as a hallmark of human speech, with troubling implications. What do philosophical reflections on bullshit have to do with algorithms? As it turns out, quite a lot.
In May this year the company OpenAI, co-founded by Elon Musk in 2015, introduced a new language model called GPT-3 (for “Generative Pre-trained Transformer 3”). It took the tech world by storm. On the surface, GPT-3 is like a supercharged version of the autocomplete feature on your smartphone; it can generate coherent text based on an initial input. But GPT-3’s text-generating abilities go far beyond anything your phone is capable of. It can disambiguate pronouns, translate, infer, analogize, and even perform some forms of common-sense reasoning and arithmetic. It can generate fake news articles that humans can barely detect above chance. Given a definition, it can use a made-up word in a sentence. It can rewrite a paragraph in the style of a famous author. Yes, it can write creative fiction. Or generate code for a program based on a description of its function. It can even answer queries about general knowledge. The list goes on.
WHO WROTE THIS?: Not long ago, hardly anyone suspected the hoax when this self-help blog post, written by GPT-3, reached the top of Hacker News, a popular news aggregation website. “You see, creative thinking is all about having fun,” GPT-3 writes. “If you spend too much time on it, then it stops being fun and it just feels like work.”
GPT-3 is a marvel of engineering due to its breathtaking scale. It contains 175 billion parameters (the weights in the connections between the “neurons” or units of the network) distributed over 96 layers. It produces embeddings in a vector space with 12,288 dimensions. And it was trained on hundreds of billions of words representing a significant subset of the Internet—including the entirety of English Wikipedia, countless books, and a dizzying number of web pages. Training the final model alone is estimated to have cost around $5 million. By all accounts, GPT-3 is a behemoth. Scaling up the size of its network and training data, without fundamental improvements to the years-old architecture, was sufficient to bootstrap the model into unexpectedly remarkable performance on a range of complex tasks, out of the box. Indeed GPT-3 is capable of “few-shot,” and even, in some cases, “zero-shot,” learning, or learning to perform a new task without being given any example of what success looks like.
Interacting with GPT-3 is a surreal experience. It often feels like one is talking to a human with beliefs and desires. In the 2013 movie Her, the protagonist develops a romantic relationship with a virtual assistant, and is soon disillusioned when he realizes that he was projecting human feelings and motivations onto “her” alien mind. GPT-3 is nowhere near as intelligent as the film’s AI, but it could still find its way into our hearts. Some tech startups like Replika are already working on creating AI companions molded on one’s desired characteristics. There is no doubt that many people would be prone to anthropomorphize even a simple chatbot built with GPT-3. One wonders what consequences this trend might have in a world where social-media interactions with actual humans have already been found to increase social isolation.
At its core, GPT-3 is an artificial bullshit engine—and a surprisingly good one at that.OpenAI is well aware of some of the risks this language model poses. Instead of releasing the model for everyone to use, it has only granted beta access to a select few—a mix of entrepreneurs, researchers, and public figures in the tech world. One might wonder whether this is the right strategy, especially given the company’s rather opaque criteria in granting access to the model. Perhaps letting everyone rigorously test it would better inform how to handle it. In any case, it is only a matter of time before similar language models are widely available; in fact, it is already possible to leverage open services based on GPT-3 (such as AI Dungeon) to get a sense of what it can do. The range of GPT-3’s capacities is genuinely impressive. It has led many commentators to debate whether it really “understands” natural language, reviving old philosophical questions.1
Gone are the days of “good old-fashioned AI” like ELIZA, developed in the 1960s by Joseph Weizenbaum’s team at the Massachusetts Institute of Technology. ELIZA offered an early glimpse of the future....
....MUCH MORE
