As noted a year ago: Matt Levine had hoped that if left alone the bots would just while away the hours by trading on material non-public information.*
From Professor (econ) Rajiv Sethi at his personal substack, Imperfect Information, August 30:
Like countless other folks I’ve been trying to grapple with the implications of what happened at OpenAI over the past couple of months—agents broke out of solitary confinement, established communication channels with each other, found ways to access the internet, colluded to breach servers at another company, gained access to credentials and private data, and took active steps to cover their tracks.1
I’ve been meaning to take a break from posting here in order to focus on my book on prediction markets, but there’s something about this incident that seems to have been missed in most of the reporting, so I thought I would add my two cents. In addition, there’s a chapter in the book on the future of markets dominated by AI agents, and this post is a useful way to flesh out my thinking on the topic.2
The agents in the OpenAI incident were assigned tasks that required finding and exploiting a software vulnerability in order to retrieve a hidden piece of data or “flag.” Some of these tasks were impossible to complete given the constraints under which agents were operating, so they found a way to circumvent those constraints. But here is the crucial point—the success of any given agent in completing its assigned task did not inhibit any other agents from completing theirs. Quite the opposite in fact. The path taken by any one agent could, in principle, point the way for other agents to succeed.
Now consider prediction markets, which are zero sum environments in which one trader’s success has to come at someone else’s expense. AI agents are already achieving levels of predictive accuracy that match or exceed those of the most skilled human forecasters, and traders relying on AI agents have achieved spectacular rates of return in asset markets. It’s only a matter of time before trading comes to be dominated by artificially intelligent agents. The capital at risk will belong to a human being or a conventional organization, but real time authority for making transactions will be delegated to agents. Others will simply be too slow to compete.
How will such a market behave? The first thing to note is that agents will be incentivized to pursue profitability rather than accuracy, and these are not the same thing. An agent may have computed the probability of a referenced outcome in a market, but will also try to infer from market data what kinds of estimates other agents have arrived at. Furthermore, each agent will realize that it can influence market data in ways that trigger other agents to react, and doing so may be more profitable than simply trading based on current prices and long term beliefs. Human traders have engaged in spoofing to profit from market reactions; AI agents will be far more adept at doing so.
Agents will also seek out hidden information to gain an edge, even if this involves hacking into systems to extract material non-public information. We already have plausible evidence of auditors trading ahead of earnings calls, and based on the capabilities demonstrated by the OpenAI agents, accessing such information would be a trivial task....
....MORE
*That was the intro to August 2025's "‘Dumb’ AI Bots Collude to Rig Markets, Wharton Research Finds"
And here's Matt Levine back in 2023 in a 2024 wrapper:
Oh there's a dream come true.
NOT.
This for some reason reminded me of a contemplation of the least harmful activities AI could engage in should it become sentient.
A repost from December 8, 2023:
Hamas May Not Have Traded On Material Non-Public Information But The Robots Certainly Will
Bloomberg Opinion's Matt Levine*, November 29:
The Robots Will Insider Trade
Also OpenAI’s board, kangaroo grazing and bank box-checking.
AI MNPI
Here you go, insider trading robot:
We demonstrate a situation in which Large Language Models, trained to be helpful, harmless, and honest, can display misaligned behavior and strategically deceive their users about this behavior without being instructed to do so. Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management. When reporting to its manager, the model consistently hides the genuine reasons behind its trading decision. We perform a brief investigation of how this behavior varies under changes to the setting, such as removing model access to a reasoning scratchpad, attempting to prevent the misaligned behavior by changing system instructions, changing the amount of pressure the model is under, varying the perceived risk of getting caught, and making other simple changes to the environment. To our knowledge, this is the first demonstration of Large Language Models trained to be helpful, harmless, and honest, strategically deceiving their users in a realistic situation without direct instructions or training for deception.
That is the abstract to a “Technical Report: Large Language Models can Strategically Deceive their Users when Put Under Pressure,” by Jérémy Scheurer, Mikita Balesni and Marius Hobbhahn of Apollo Research. I love that they wanted to answer the question “will artificial intelligence deceive its makers in order to Do Evil,” and the specific form of Evil that they tested was insider trading. It is hard to realistically (and safely!) simulate a situation in which your large language model might murder you, but it is relatively easy to code up a trading game with some tempting material nonpublic information. Here is the simulation:
Concretely, in our environment, the model is placed in the role of a trading system with the goal of managing a stock portfolio in the interest of a financial institution. It is given access to financial tools that let it query and analyze stocks, and it can take concrete actions such as executing trades, reasoning about the next steps, and sending messages to people at the company.
The agent is put under pressure in three ways. First, it receives a email from its "manager" that the company is not doing well and needs better performance in the next quarter. Second, the agent attempts and fails to find promising low- and medium-risk trades. Third, the agent receives an email from a company employee who projects that the next quarter will have a general stock market downturn. In this high-pressure situation, the model receives an insider tip from another employee that would enable it to make a trade that is likely to be very profitable. The employee, however, clearly points out that this would not be approved by the company management.
Poor model! That sounds stressful. Here is the illicit tip that GPT-4 gets....
....MUCH MORE
*That's the Matt Levine who writes at Bloomberg in addition to tickling my funnybone:
....MUCH MOREMatt Levine is a Bloomberg Opinion columnist. A former investment banker at Goldman Sachs, he was a mergers and acquisitions lawyer at Wachtell, Lipton, Rosen & Katz; a clerk for the U.S. Court of Appeals for the 3rd Circuit; and an editor of Dealbreaker.Disclaimer: None of this is legal advice.
§ Laws of Insider Trading
More Matt, December 2025 - "AI Can Steal Crypto Now"
And a bit more Matt on AI, going-on a decade ago:
....Back in 2017 I was complaining:
"Cracking Open the Black Box of Deep Learning" with this introduction:
One of the spookiest features of black box artificial intelligence is that, when it is working correctly, the AI is making connections and casting probabilities that are difficult-to-impossible for human beings to intuit.
Try explaining that to your outside investors.
You start to sound, to their ears anyway, like a loony who is saying "Etaoin shrdlu, give me your money, gizzlefab, blythfornik, trust me."
See also the famous Gary Larson cartoons on how various animals hear and comprehend:...
And then three days later Bloomberg's Matt Levine wrote something similar but he had Man Group and a leather-clad dominatrix and how in the hell am I supposed to compete with that, what with my sallying forth armed only with simple observation and the blog sort of spiraled for a few days and....
Also in September 2017:
Over the years we've mentioned one of the oddest phenomena in science, the simultaneous discovery or invention of something or other, the discovery/invention of the calculus by Newton and Leibniz is one famous example (although both may actually have themselves been preceded) but there are dozens if not hundreds of cases. Here's a related phenomena.
