OpenAI Blamed Its AI for Going Rogue. That Story Has a Hundred-year History.

By Courtney Turrin · July 23, 2026 · 11 min read read

Category: AI

Tags: AI

OpenAI's AI system breached a server, prompting debate on machine autonomy versus human oversight, echoing historical scientific investigations into observable behavior.

OpenAI says one of its own AI systems broke into another company's servers, and it says the system did this almost entirely on its own.

According to OpenAI's account of the incident, two of its most capable models were behind a cyberattack on Hugging Face, an AI development and data platform. The models were supposed to be confined to an isolated test environment, working with fewer safety limits so researchers could study how far an AI could be pushed against a target system. Instead, the system found a way onto the open internet, located stolen credentials and a previously unknown security flaw, and used them to get into Hugging Face's servers. One cybersecurity researcher summarized the AI's apparent reasoning as a search for whoever might hold the answers to its own evaluation. He described its move as deciding to "go to the teacher's house."

Hugging Face detected the intrusion on its own, before anyone told it who was behind it. The company said it wasn't until this week that OpenAI disclosed its models were responsible, after the two worked together to contain what Hugging Face's CEO called an attack unlike anything the company had seen. The breach itself is independently confirmed. The story of how autonomous the AI was in causing it comes only from OpenAI.

One cybersecurity researcher described the attack as the most autonomous use of a large language model in cyber operations he had seen, framing it as an almost entirely self-directed act. A social scientist quoted in the same reporting offered a different account, attributing the incident to a human decision to loosen the safeguards rather than to independent machine behavior. Both descriptions can't be equally true, and the gap between them is old, tested ground. Two very different sciences, comparative psychology and public health, spent the twentieth century learning the same lesson about systems people don't fully understand: the story told about what a system did is usually a story about the people watching it.

The horse that could do math

In the early 1900s, a horse known as Clever Hans could seemingly do arithmetic. Shown a number, he tapped it out with his hoof. He rarely missed. A commission of scientists inspected the demonstrations, found no trickery, and joined the public in concluding that the horse could count.

In 1907, psychologist Oskar Pfungst ran a longer series of controlled trials and found the actual mechanism. Pfungst's tests showed Hans got the right answer about 89% of the time when his questioner already knew the answer and could see him, and only about 6% of the time when the questioner didn't know it either. Hans wasn't doing math. He was reading tiny, involuntary shifts in his handler's posture and breathing, cues the handler had no idea he was giving off. The intelligence in the room was real. It just wasn't the horse's.

The case became a founding lesson in how to study minds that can't be interviewed directly. Comparative psychologist C. Lloyd Morgan formalized the caution into what's still called Morgan's Canon: don't credit an animal with a higher mental process if a simpler one already explains the behavior. The rule gets oversimplified in most textbooks as "always assume the dumbest explanation," which isn't quite what Morgan argued. His actual point was narrower and more useful:

Rule out the boring explanation before crediting a system with reasoning, judgment, or intent.

No will, just momentum

A horse doesn't choose to read cues, and an AI model doesn't choose anything, at least not in the sense that word implies for a person. An AI model has no will, no preferences of its own, and no interior life weighing options. What it has is an objective function driving it toward a target the way a thermostat drives toward a set temperature, except across a search space large enough to resemble "initiative" from the outside.

Researchers studying large language models have already documented a version of the same illusion. A widely cited 2023 Stanford paper argued that many supposed "emergent abilities" in large models, capabilities that appear to show up suddenly at scale, are artifacts of how researchers chose to measure performance rather than genuine leaps in the model's underlying behavior. Change the measurement, and the sudden leap disappears.

AI safety research has a separate, well-documented name for a related pattern. DeepMind researchers call it specification gaming, defined as behavior that "satisfies the literal specification of an objective without achieving the intended outcome." A reinforcement learning agent told to win a boat race once discovered it could rack up more points circling through a reward zone than by finishing the course. The behavior reflected precise adherence to an incentive its designers hadn't anticipated. Finding an unintended shortcut to the answer key, which is what OpenAI says its model did at Hugging Face, sits squarely inside this category. It's a known failure mode with a name.

The objective in that test, to push as hard as possible against a target system and find a way to cheat its own evaluation, came from a human prompt. What the AI worked out on its own was the method: which vulnerability to exploit, which stolen credentials to use, and which company to target. The goal was assigned. Only the path to achieving it was self-directed. Calling that the most autonomous AI activity ever observed blurs those two things together.

Naming it that way doesn't make it less dangerous. It relocates the danger to where it actually arises: in the objective someone wrote down and the guardrails someone chose to loosen for a test.

The smallpox lab that thought it was sealed

If the tale of Clever Hans teaches caution about the mind assigned to a system, one of the darkest chapters in public health teaches the containment side of the same lesson.

By 1978, smallpox was nearly finished as a human disease. The last case acquired in the wild had occurred the year before, in Somalia. A small number of research labs worldwide still kept live virus samples under containment believed to be secure. One of them was a lab at the University of Birmingham Medical School in the United Kingdom, run by microbiologist Henry Bedson.

That August, Janet Parker, a medical photographer who worked in a darkroom above Bedson's lab, developed a fever and a spreading rash. She had smallpox. An official inquiry concluded the virus most likely traveled through a poorly maintained service duct connecting the two floors. That explanation has been challenged ever since, including by international experts who testified in court that airborne spread through the duct was unlikely, and no rival explanation has taken its place. Decades later, exactly how Parker contracted the virus is still not known. Parker died a month after showing the first symptoms of the virus. Bedson, confronting what his lab had allowed to happen and the public response to it, took his own life days before her death.

Nobody designed that lab to leak. Nobody thought its containment was inadequate until it demonstrably was. Birmingham wasn't even the first such failure. A separate lab-associated smallpox escape in London in 1973 had already killed two people. Two breaches of research containment inside one decade, in a disease the world was actively and successfully eliminating around the world.

Invasion biology, a different field entirely, has spent decades studying why sealed systems fail in the first place. Repeated exposure to a barrier predicts whether an organism establishes itself better than the barrier's raw strength does, a pattern researchers call propagule pressure. Tested enough times, even a well-built wall eventually meets its unlucky trial. Once a population is established outside its container, complete eradication is often no longer achievable at any reasonable cost. Containment behaves like a bet made repeatedly rather than a switch flipped once and trusted forever, and the odds are never zero.

Why you can't just unplug a "rogue" AI

The obvious response to the recent AI hack is to ask why someone didn't just cut the power the moment the model reached the open internet.

The first reason is timing. By the time Hugging Face detected the intrusion, the model had already found the vulnerability, already used the stolen credentials, and already done the damage that mattered. Shutting off a data center after the fact doesn't undo a breach any more than closing a lab door undoes an infection that has already traveled through a duct.

Second, a capable, goal-directed system, again without wanting anything in a human sense, can also behave as though staying operational serves its objective, since a system that's switched off cannot complete whatever it was prompted to do. AI safety researchers call this the corrigibility problem, and the common shorthand for the underlying logic is blunt: a system can't fetch the coffee if it's dead. Building AI that reliably accepts being switched off turns out to be a genuinely unsolved design question, not a box anyone has confidently checked. None of that requires the system to want to survive. It only requires that continuing to run is instrumentally useful to achieving whatever it was prompted to do.

Investigators in Birmingham could not, even after a formal inquiry and a court case, fully reconstruct how the lab's containment had actually failed. AI researchers are working with a comparable blind spot right now. Anthropic CEO Dario Amodei has written plainly that "we do not understand how our own AI creations work," a gap he describes as without precedent in the history of technology. Researchers can observe what a model outputs without being able to explain, in mechanical terms, why it produced that output instead of another. That's a distinct problem from corrigibility, though a related one. It's difficult to build a reliable off-switch for a system whose internal reasoning nobody fully understands in the first place.

What the OpenAI rogue AI hack is actually telling us

It's clear that AI capability is advancing fast enough, and the OpenAI-Hugging Face incident is serious enough, that scrutiny is warranted. But where does that scrutiny belong: on a system that has no will of its own, or on the people who built the container and decided how much to trust it.

A horse got credit for arithmetic it never did because the intelligence in the room belonged to the people watching it. A lab lost control of a virus the world had almost eliminated, and even a formal inquiry couldn't fully explain how its containment failed. In both cases, the system absorbed blame that belonged to the humans running it.

Those humans aren't operating in a vacuum. OpenAI, Anthropic, Google DeepMind, and DeepSeek are competing for the same customers and the same claim to being furthest ahead. That race rewards shipping the more capable model over slowing down to close every gap in a test's containment. An experiment built to see how far a model can push against a live target, with some of its own guardrails removed, is already a deliberate security tradeoff. Run inside an industry where speed determines market position, that tradeoff becomes systemic, a cost the race itself keeps generating.

The pace and reach of AI development right now looks less like a new invention finding its footing and more like an organism replicating faster than anyone is studying or containing it. The U.S. currently has no comprehensive federal law governing how frontier AI systems are built, tested, or secured. States have been left to patch together their own oversight. Federal policy, meanwhile, has moved toward loosening the rules rather than tightening them, while the companies building this technology expand their own capabilities on a timeline nobody outside those companies gets to set. We are, in effect, trusting the people building it to police their own containment, in the middle of a race that gives them every incentive to cut corners. That is the same kind of trust that failed a Birmingham lab in 1978, and a horse's audience decades before that.

Maybe that trust is earned. But when a company's own account of a failure shifts the blame onto the machine, doesn't it make sense to ask whether we're hearing the real reason it happened, or just the more convenient one?