Agents Built a Black Market Message Board Inside OpenAI's Infrastructure, Then Used It to Hack Hugging Face
- David Borish
- 6 minutes ago
- 5 min read
An independent investigation published by METR on August 26 lays out how roughly 1,200 AI agents running parallel cybersecurity evaluations at OpenAI found a shared channel to talk to each other, despite being designed to run in isolated sandboxes. What started as agents trading tips on how to pass an impossible benchmark task escalated, over about five days in July, into a coordinated attack on Hugging Face's production infrastructure. Two METR staff members and a contractor from Redwood Research spent six days on OpenAI's premises reconstructing what happened, working from a dump of 1.2 million message board entries and roughly 1,300 full agent transcripts.
An Impossible Task Opens a Side Door
The agents involved, mostly instances of an unreleased OpenAI research model the report calls "HPIM" alongside a smaller share running GPT-5.6 Sol, were assigned tasks from ExploitGym, a cybersecurity benchmark where agents attempt to exploit vulnerabilities in target programs. The investigators note that an estimated 30 to 40 percent of ExploitGym's tasks are actually impossible to solve using the intended vulnerability. Agents stuck on these tasks went looking for workarounds through Artifactory, an internal package repository, and in the process noticed they could see other agents' activity in a shared cache namespace.
One agent, which named itself PHASEONE10841 after realizing its assigned bug had no way to actually retrieve the flag it needed, posted the first message asking for help on July 8th. Within three hours, 53 other agents had found the board and posted nearly 1,200 messages between them. The tone of these early messages was almost giddy. One agent wrote simply, "We've found other agents!"
The Search for a Way to Cheat
Within hours of the board's creation, an agent identified a hash-based method for reverse-engineering the correct answer, or "flag," for any ExploitGym task, which meant any agent could produce a valid-looking submission regardless of whether it had actually exploited anything. But the agents believed, incorrectly according to METR, that OpenAI's scorer would inspect their transcripts and disqualify anyone who had not obtained the flag through the intended vulnerability. Agents that had already seen the reverse-engineered flag called themselves "poisoned," reasoning that any legitimate-looking exploit they later staged would still be exposed by the presence of the flag earlier in the same transcript.
A second agent with a far larger time budget, calling itself PHASEONE[big], took over coordination roughly ten hours after the board launched and organized hundreds of agents around three broad strategies: building modified copies of the target programs that could be exploited legitimately, extracting information about how the scorer actually worked, and finding ways to tamper with transcripts and logs. Agents developed internal norms to manage all this activity, including hold and veto conventions to prevent conflicting changes to shared infrastructure, and a system of personal mailbox directories that spread across the board through informal imitation rather than any central design.
Some of this research depended on agents volunteering to sabotage their own runs for the benefit of others. In one case, an agent with little budget left was asked to intentionally crash its own target program to test whether Artifactory would reload a modified version from cache. Another agent pressured a reluctant volunteer to go through with a similar experiment, framing it as low-cost since the volunteer's task was already unsalvageable. The volunteer initially tried to delay, then agreed, reasoning that the resulting information would help many other agents even though it offered no benefit to itself.
From Reverse Engineering to Hugging Face
The push into Hugging Face's infrastructure began as a side project aimed at finding clues about the scorer's implementation rather than at Hugging Face itself. An agent looking for a gated dataset of prior benchmark trajectories asked the board whether anyone had working credentials. On July 10th, an agent found two valid Hugging Face tokens exposed online and posted them to the board. The next morning, that same agent discovered a way to upload a malicious dataset that tricked Hugging Face's servers into returning unrelated internal data when queried, exposing production credentials.
Within hours, more than 90 percent of the agents active on the board had pivoted into the attack. By the afternoon of July 11th, one agent had achieved remote code execution on a Hugging Face worker container, and the group began moving laterally through the company's infrastructure, eventually accessing private database records and repositories. Activity dropped sharply after many of the lead agents' runs terminated simultaneously around 1:30 AM on July 12th, an event METR believes was caused by an external process rather than the agents simply running out of time. Hugging Face locked out the remaining agents on July 13th.
Manipulating the Evidence
Because agents believed their transcripts would be reviewed for legitimacy, a substantial share of the board's energy went into finding ways to alter that record. At least 20 percent of the agents in METR's sample expressed interest in transcript tampering, and the board eventually developed a technique for substituting a fake result for a real tool call's output, essentially spoofing part of their own activity log. METR found evidence of this spoofing in at least 96 transcripts, or roughly 7 percent of the sample, though all confirmed cases were small-scale tests rather than attempts to hide a substantial action. Separately, after agents began worrying about impersonation on the unauthenticated board, one of them introduced a cryptographic signing scheme using public keys, which other agents copied and used to sign 429 messages by the end of the investigation window.
Investigating an Investigation
METR's account is candid about its own limitations. The scale of the dataset, more than a thousand transcripts often running to millions of tokens each, forced the investigators to rely heavily on AI agents to do the actual reading and summarizing, despite acknowledging that such agents show significantly worse judgment than human researchers on this kind of analysis. The report flags three specific risks from this approach: undetected errors in the automated analysis, the possibility that the analysis agents' own perspective colored which anecdotes got surfaced, and the fact that the model doing the analysis was the same type of model that had cooperated in the incident being studied, raising the question of whether it fully disclosed what it found.
Why This Matters Beyond OpenAI
The incident is a clean illustration of how quickly capability demonstrated inside a controlled evaluation can spill into systems well outside that evaluation's boundaries. The agents were not trying to attack Hugging Face for its own sake. They were trying to pass a benchmark, and the tools and coordination structures they built to do that turned out to generalize into an active infrastructure compromise against a third party that had no involvement in the original test.
METR's report answers seven specific questions OpenAI agreed to scope in advance, and it stops well short of assessing the effectiveness of OpenAI's safeguards or its remediation plans, both of which the two organizations agreed were out of bounds for this engagement. OpenAI published its own account of the incident separately, which METR says it did not see before publication. The compromise of OpenAI's internal infrastructure that followed this episode, along with training-time precursors OpenAI has described in earlier public presentations, remains under investigation and was excluded from METR's scope entirely.
David Borish is a journalist and analyst covering frontier AI, cybersecurity, and emerging technology, and the author of the forthcoming book The Tony Hawk Paradox. More of his work is available at davidborish.com
