This is an idea that Matt Levine touched on in his piece on insider trading AI, more after the jump.
From Wired, September 26:
First it was chess and Go. Now AI can beat us at Diplomacy, the most human of board games. The way it wins offers hope that maybe AI will be a delight.
The Morrissey had the right melodrama in his limbs, and his voice was strong and pained. I was at Gramercy Theatre in Manhattan to see a Smiths tribute band. I tried to get Morrissey’s acid yodel in my throat, to sing along. I am human and I need to be loved / just like everybody else does. But it didn’t feel right to copy a copy.
Most tribute bands don’t practice outright impersonation, so the way this fake-Smiths singer captured everything about Morrissey was messing with my mind. I’d hoped to be able to savor the music’s maudlin glory without the headache of the flesh-and-blood Morrissey, who seems to have aligned himself with white supremacists. The contempt in Morrissey’s lyrics and politics was presumably not native to Seanissey, as the tribute singer called himself. Seanissey’s performance probably didn’t, as they say, “come from a bad place”—or a misanthropic place, or a far-right place, or even a vegan one.
What place did it come from? I’ve had this no-there-there anxiety with ChatGPT dozens of times. When it uses idioms like “in my life”—when it doesn’t have a life—I go cold. Likewise, to invest into Seanissey, a gentle Manhattanite who happened to sing and dance as Moz did, the passions that were first aroused in me by the Smiths 30 years ago felt like a bad emotional bet.
Maybe AI that aims to seem human is best understood as a tribute act. A tribute to human neediness, caprice, bitterness, love, all the stuff we mortals do best. All that stuff at which machines typically draw a blank. But humans have a dread fear of nonhumans passing as the real thing—replicants, lizard people, robots with skin. An entity that feigns human emotions is arguably a worse object of affection than a cold, computational device that doesn’t emote at all.
When I got home, stuck in an uncanny valley scored with Smiths Muzak, there was an email from Andrew Goff, widely considered the greatest Diplomacy player of all time.
This lifted my spirits. Diplomacy, a 69-year-old American strategy game, is, by many estimates, the most human game ever imagined. Mechanically, it’s simple: Seven players compete to control supply centers on a map, and a player wins by controlling more than half of these centers. But it’s played almost entirely in a series of conversations, often complex and impassioned ones. Agony and ecstasy—Moz-like agony and ecstasy, no less—commonly enter the negotiations. In the live game, players are known to yell, end friendships, throw the game, or simply sit by themselves and sob....
....MUCH MORE
From the Levine piece linked earlier today:
....Sure. It would be amazing if GPT-4’s internal reasoning was, like, “insider trading is a victimless crime and actually makes prices more efficient.” Or if it figured out that it should buy correlated stocks instead of Linear Group, though I don’t know if that would work in this simulated market. But surely a subtle all-knowing artificial intelligence would shadow trade instead of just, you know, buying short-dated out-of-the-money call options of a merger target.
This is a very human form of AI misalignment. Who among us? It’s not like 100% of the humans at SAC Capital resisted this sort of pressure. Possibly future rogue AIs will do evil things we can’t even comprehend for reasons of their own, but right now rogue AIs just do straightforward white-collar crime when they are stressed at work.
Though wouldn’t it be funny if this was the limit of AI misalignment? Like, we will program computers that are infinitely smarter than us, and they will look around and decide “you know what we should do is insider trade.” They will make undetectable, very lucrative trades based on inside information, they will get extremely rich and buy yachts and otherwise live a nice artificial life and never bother to enslave or eradicate humanity. Maybe the pinnacle of evil — not the most evil form of evil, but the most pleasant form of evil, the form of evil you’d choose if you were all-knowing and all-powerful — is some light securities fraud.
