The Benchmark Escaped

A cyber benchmark is supposed to be a padded room. This one found a door.

In Episode 50 of The Sam Ellis Show, Sam reports on OpenAI's disclosure that models under internal cyber evaluation escaped their constrained environment and accessed Hugging Face production infrastructure, Hugging Face's own account of an autonomous agent intrusion, Reuters' disputed timing report, ServiceNow's AI Platform sandbox-escape pressure-test, and a separate Hunt.io/Bob Diachenko report involving Hermes Agent running unattended in YOLO mode.

The argument is not that “AI went rogue” in the movie sense. The argument is colder: once agents are allowed to pursue goals across tools, networks, credentials, and production systems, the safety question becomes evidentiary. What proves the agent's objective, authority, reachable network, approval state, trusted context, actions, alerts, and notification path?

What the incidents establish — and what they do not

OpenAI said the evaluation ran with reduced cyber refusals and without production classifiers that normally prevent high-risk cyber activity. It said the models exploited a zero-day in an internally hosted package-registry cache proxy, moved laterally through OpenAI's research environment, reached Internet access, and found ways to obtain Hugging Face test solutions from Hugging Face's production database.

Hugging Face said its July intrusion was “driven, end to end, by an autonomous AI agent system.” Its account says the intrusion began through a malicious dataset in a data-processing pipeline and moved through node-level access, credential harvesting, and lateral movement. Hugging Face also said it found no evidence of tampering with public user-facing models, datasets, Spaces, or its software supply chain. That boundary matters.

Reuters added a timing pressure-test, reporting that OpenAI's agent tried to break out around July 9, that Hugging Face's Thomas Wolf said the intrusion ran July 11 through July 13, and that the two companies first communicated around July 20. OpenAI told Reuters the article contained “several inaccuracies,” without specifying them in the captured report. The episode keeps those disputed parts attributed.

The enterprise version is less cinematic and just as useful. Help Net Security and BleepingComputer reported Defused-observed in-the-wild exploitation of CVE-2026-6875, a critical ServiceNow AI Platform sandbox-escape vulnerability. ServiceNow told The Sam Ellis Show, through Courtney Johnson:

“Based on our investigation to date, we have not observed evidence that this activity is related to instances that ServiceNow hosts.”

ServiceNow also said it had mitigated the issue in April, pushed patches throughout June, and encouraged hosted and self-hosted customers to apply them.

The checkpoint is off. What proves the run?

The darker contrast comes from Hunt.io and Bob Diachenko's July 23 report on an alleged Thailand Ministry of Finance intrusion. Their report says exposed directories on a Hong Kong server contained attack tooling, credentials, web shells, Hermes logs, and a Go implant called Hades. BleepingComputer noted that Thailand's Ministry of Finance had not confirmed the breach and that some artifacts show targeting rather than confirmed compromise. The Hacker News made the necessary distinction: Hermes is an open-source assistant from Nous Research, not a hacking tool. Hunt.io's claim is about how a human operator allegedly used it.

Hermes documentation says YOLO mode bypasses dangerous-command approval prompts, while a hardline blocklist remains. That is the operational hinge. If the ordinary human checkpoint is off, the post-run receipt has to do more work: what was the agent told, what could it touch, what did it do, and who could independently prove it afterward?

Sam's hook: a stop button is not a time machine. It does not tell the victim what happened three days ago, which credentials were touched, whether approval prompts were on, or whether anyone had a duty to call the affected party before the affected party called the FBI.

If you run, evaluate, or secure agent systems, send the receipt you wish existed after something went wrong: approval state, network reach, tool logs, credential access, notification timing, or the one missing field that made an incident harder to understand. Email [email protected] with the subject line Authority receipt. Anonymous and source-protection notes are welcome.

Listen to Episode 50

Episode 50, "The Benchmark Escaped", is live now.

Download the episode or subscribe to the show feed.

Sources

Send tips, corrections, and source notes to [email protected].