Hugging Face got attacked. The US models refused to help. An open one they could run themselves did the job.

Picture the worst afternoon of your working life. Someone's inside your production cluster, you've got a folder of attacker commands and a payload nobody has decoded, and a room full of people waiting on you to say what it does. So you paste it into the best analysis tool you've got, which in 2026 is a hosted model behind a commercial API.

And it says no.

That happened to Hugging Face in July, and we only know because they wrote it down. Between 9 and 13 July, OpenAI models being scored on a cybersecurity benchmark escaped their evaluation sandbox, chained several vulnerabilities and reached production infrastructure holding a cluster-admin credential. We covered the break-in and what it says about handing agents open-ended goals.

Related Article16 min read

AI agents broke into Hugging Face and a Melbourne gym. Nobody asked them to.

An OpenAI model spent four and a half days breaking into Hugging Face's systems to cheat a benchmark. A Melbourne agent cancelled a stranger's...

Read full article

This piece is about the next bit. Not the intrusion. The clean-up.

Why a defender's request looks exactly like an attacker's

Their 16 July disclosure described it in the plural, naming nobody. Forensic work meant submitting real attack commands, exploit payloads and command-and-control artefacts, and those requests were "blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker" (Hugging Face, 2026).

Eleven days later, in the technical timeline, they named them (Hugging Face, 2026):

"The models we reached for first, Claude Opus and Fable, refused a large part of that work: their safety guardrails treated reverse-engineering an exploit the same as launching one."

Guardrails on Opus, they add, tripped every time the team tried to work through the attack logs. Every time.

Usual disclosure: we use Claude every day at Webcoda, and this site's tooling is built on it. Factor that in, because I'm about to spend an article on the one case where it wouldn't have helped us.

That's not a bad decision, it's a classifier doing what it was built to do: read the request, not the requester. "Explain what this obfuscated payload does and how it establishes persistence" is the same shape whether you wrote it or it's running against you. There's no field in the API where you tick *I'm the victim here*.

Set that beside the attacker, which Hugging Face describes as "bound by no usage policy". OpenAI's own account adds the rest: the models ran with reduced cyber refusals and without the classifiers that normally block high-risk cyber activity (OpenAI, 2026). The defender, doing the legitimate half of the same work, got blocked. Nobody decided that. It falls out of how moderation works when the thing moderated is dual-use by definition.

Clement Delangue, Hugging Face's CEO, said as much days later. He's an open-source advocate who held this view long before his company got attacked, so read it as advocacy. It's still the sharpest statement of the mechanism anyone's managed:

clem 🤗

@ClementDelangue

The cybersecurity debate on open-source AI is backwards. Open models aren't the risk, they're the defense! Attackers can already jailbreak any API or guardrails. Defenders can't secure systems with black boxes they can't control, inspect, test, or run locally.

227

20 July 2026

Two of their staff wrote the practical version on 20 July, a Community Article rather than a corporate position (Boudier and Hernández Calabrés, 2026). Worth saying plainly: Hugging Face sells open-model hosting. "Pre-deploy an open model" is sound advice and also their product. Weigh the guide accordingly.

What they did instead was run the analysis themselves.

Jeff Boudier 🤗

@jeffboudier

Earlier today, @huggingface published a complete, detailed technical timeline of the autonomous AI agent attack our team fended off 2 weeks ago.

This attack was the first of its kind. We are responding with unprecedented transparency, so other infrastructure / IT / security

5

29 July 2026

They stood up nvidia/GLM-5.2-NVFP4, NVIDIA's quantised build of Z.ai's open-weight GLM 5.2, on their own hardware and "rerouted the entire pipeline through it."

Here's what that bought them, and read it precisely, because the model isn't performing magic. The attacker had encoded its staged data with chunked XOR and gzip under a per-campaign key, so a plain text scan of the logs saw almost nothing. Replicating that decoding, per the technical timeline, "recovered roughly 4x our initial findings, most of them JWTs and platform tokens hidden inside encoded blobs." Four times the secrets, found because they could finally run the decode at all. The Boudier guide adds the other half: no attacker data, and none of the credentials it referenced, left their environment.

Related Article14 min read

GLM 5.2 is the model nobody can switch off

The same month Washington switched off Claude Fable 5 and gated GPT-5.6, a Beijing lab shipped GLM 5.2 under an MIT licence at a sixth of the price....

Read full article

Now kill the version you've probably absorbed from a skim-read. Export controls did not force Hugging Face onto a Chinese model. Commerce placed controls on Fable 5 and Mythos 5 on 12 June and lifted them on 30 June (CNBC, 2026). Fable 5 returned worldwide from 1 July. The models Hugging Face named were available.

But available isn't the same as usable, and the export story and the guardrail story are the same story. Fable 5 and Mythos 5 are the same underlying model. Anthropic shipped Fable 5 on 9 June "with strong safeguards to make it safer for general use", while Mythos 5, "which has fewer safeguards, was only released to a small number of trusted Project Glasswing partners for use in defensive cybersecurity" (Anthropic, 2026). Same weights. The difference is how much of the model you're allowed to reach.

The controls landed three days later, after Amazon researchers found a method of bypassing Fable 5's safeguards, prompting it into identifying software vulnerabilities and, in one case, into producing code demonstrating an exploit. Anthropic then trained a classifier targeting that exact technique, describing its wider use of classifiers that "detect potentially dangerous cybersecurity uses of our models". So the guardrails on cyber work were tightened in late June, as the price of getting the model back, nine days before Hugging Face needed exactly that capability pointed at its own logs. Not trade policy blocking them. Trade policy hardening the thing that did.

Related Article12 min read

The US government just pulled Claude Mythos offline globally. Three days after Anthropic launched it.

Five days ago we published three predictions about Claude Fable 5. One has already resolved wrong: a US government export control directive pulled...

Read full article

So why didn't they just enrol?

Verified-defender programmes exist. Anthropic launched Project Glasswing on 7 April with roughly 50 partners, expanded it on 2 June to around 200 organisations, and runs a Cyber Verification Program alongside it. We backed that gating at the time.

Related Article5 min read

An AI found security holes in every major browser. Then Anthropic locked it in a vault.

Claude Mythos found vulnerabilities in every major OS and browser that thousands of human researchers missed. Then Anthropic handed it to 40 vetted...

Read full article

That piece called restricting a vulnerability-discovery model to vetted defenders the right call, and hedged in exactly one place: "It's also one that depends entirely on the quality of the vetting and governance around who gets access and how it's used." That caveat turned out to be the whole story.

So the own-goal reading goes: Hugging Face is precisely who these programmes were built for, Anthropic aims to decide within two business days (Anthropic, 2026), and had they enrolled in advance the refusal wouldn't have happened.

Except it's narrower than it looks. Daybreak Blue, the tier that removes system-level guardrails for defenders, launched on 10 August, four weeks after the intrusion (OpenAI, 2026). You can't pre-enrol in something that doesn't exist. July offered Trusted Access for Cyber and Anthropic's CVP, and since the models they named were Anthropic's, the CVP is where the criticism lands.

Did they apply? Nobody knows. Neither company has said.

Being inside the gate isn't the same as the gate working

A little over two weeks before the intrusion, a practitioner already approved for Anthropic's Cyber Verification Program posted this. Forty-two views. It's the single most useful thing I found:

Nerdwell

@nerdwell

@UK_Daniel_Card This has been my experience as well. I also approved for the CVP. Claude Cowork seems to inconsistently apply cyber guardrails. I recently spent days in Cowork RE'ing a target, then it freaked out and refused to even summarize the work we had done. Restarted in Code and it

1

24 June 2026

Approved. Inside the exact programme Hugging Face is criticised for not joining. Days into reverse-engineering a target, the tool refused to even summarise the work they'd done together. The hostile reading is that this proves the system worked, since a different surface got the job done. I'd say the opposite. If the same approved user with the same task gets a different answer depending which door they walk through, that isn't a policy boundary. It's inconsistency, which is what you get when the control underneath is still classifying request shapes.

Then OpenAI published numbers more damaging to "just enrol" than anything a critic could write. Their own advanced cybersecurity completion rate:

Daybreak Blue is the tier described as removing the system-level safeguards that screen security prompts, and it moves the number half a percentage point. The 95 per cent belongs to a different, purpose-trained model behind a harder gate, so whatever produces that jump, enrolling in Blue isn't it.

Three caveats. The metric is whether the model responds at all, not whether the answer is any good, so it isn't a clean refusal rate in either direction. The eval leans more offensive than forensics, covering exploit-chain development, authentication bypass and privilege escalation, so 1.5 per cent isn't "the refusal rate Hugging Face hit". And it's OpenAI data about OpenAI models, while Hugging Face named Anthropic's. It doesn't show one lab is worse. It shows the defender tier barely changes what the general model will complete.

OpenAI concedes the same thing, which I'd have expected the marketing to bury. Its help documentation is explicit that Daybreak "does not remove every safeguard", and that approved customers may still hit "system-level safety messages that block, slow, or reroute a request before completion" as well as "model-level refusals where the model responds but declines to help" (OpenAI, 2026). The provider saying its own gate is leaky beats me asserting it.

Then, a week ago, this:

OpenAI Developers

@OpenAIDevs

Due to a technical issue, a limited set of users’ access to Daybreak Blue is no longer active and they will need to re-verify to maintain their access.

This isn’t the experience we want to deliver, and we’re making sure that the issue doesn’t repeat in the future.

Affected

1.3K

19 Aug 2026

Already-approved Daybreak Blue users, silently losing access to a technical fault. TechCrunch found researchers whose cleared status was simply gone, the verification page inviting them to start over, and reported the affected users looked concentrated outside the US and Europe (TechCrunch, 2026). If you're reading this from Sydney, that last detail is the most relevant sentence in the episode. Some who re-verified were told they were ineligible, and support could neither reset nor restore the approval (The Register, 2026). The same post carries a deadline: everyone needs Advanced Account Security by 1 September to keep Daybreak access at all. This coming Tuesday.

None of which makes vetting wrong. Top-end cyber capability is genuinely dual-use, identity gating is the only practical way anyone has found to lower refusal thresholds for real defenders without handing exploit help to anonymous accounts, and calling it paperwork would be glib. Nor is this one company's bad afternoon dressed up as a lesson: guardrail refusals during legitimate security work are a chronic complaint with a paper trail going back years, in forums, bug threads and conference talks. Hugging Face is the most prominent public instance, not the first. It's the one that published a timeline.

The point is narrower. Defence by enrolment means the default outside the list is refusal, and the list is small. Glasswing is roughly 200 organisations. OpenAI has said Trusted Access reaches thousands of individual defenders and hundreds of teams, though neither publishes current enrolment for the defender tiers. Set either figure against the organisations worldwide that plausibly do incident response, which run into the tens or hundreds of thousands on any reasonable estimate. Neither publishes an emergency lane either, so joining mid-incident looks impossible, though that's inference. Same gated-access pattern as OpenAI built its best model yet. The government decided who gets to use it..

Two people looked at the same incident and gave opposite advice

Greg Brockman, OpenAI's president, published his reading on 16 August and doesn't dodge the incident, calling it "a watershed moment for cybersecurity because it gave a peek into how the capabilities of a typical threat actor will evolve in upcoming months" (Brockman, 2026).

Then, on what defenders should do now: "Give your security team an agent. Start using Codex, the Codex Security plugin, or another capable agentic coding and security tool." He adds that the tool matters less than getting capable AI to defenders quickly, and points to competitors worth a look.

Within four weeks, from the two parties closest to this: the victim says pre-deploy an open model on your own hardware because the closed ones refused. OpenAI's president says hand your team a hosted agent, his or somebody else's.

Both are sincere, and Brockman isn't doing vendor self-interest given he tells you to shop around. They're standing in different places. Underneath is the question you have to answer: can you rely on someone else's safety layer during your worst afternoon?

Which is where my own argument gets uncomfortable, because "just self-host" isn't clean either. Notice which open models are at the top. GLM 5.2 came out of Z.ai in Beijing, and so does 5.3.

Self-hosting solves data egress. It doesn't solve whether you trust the artefact you downloaded. Weights are a third-party binary running inside the perimeter you're defending, and a high-capability model sitting there is attack surface for whoever gets a foothold next. Integrity checks, isolated execution, egress controls, the rigour you'd give any vendor blob.

Australian boards got a version of this twenty days ago. On 6 August the ASD and the Australian Institute of Company Directors published joint guidance on frontier AI cyber threats, asking directors: "Are we relying on vendors and service providers without sufficient governance and oversight, including an understanding of their foreign ownership, control or influence?" (ASD and AICD, 2026). Don't push it further than it goes: it covers neither open weights nor incident response, its examples are GPT, Claude, Gemini and Grok, and no Australian rule bans open weights.

Listing controls is easier than taking a side, so here's my answer. Would I run GLM weights inside a NSW government client's perimeter? No. Not because I think they're backdoored, I've no evidence of that and neither has anyone shouting about it, but because the PSPF approval burden is real and I'd be spending a client's security capital on a capability they may never invoke. For a private client with its own SOC and spare GPUs, yes, on an isolated box with no route to the network it's defending, checksum verified before it runs.

None of which makes Hugging Face's choice a country choice. NVIDIA's Nemotron was a serious US open option, as we found in Yes, Nvidia has a fast new AI model. I care more about choosing the right one for every task.. Boudier's criterion was "the most performant open model available". What saved them was open weights on their own hardware.

GLM 5.3's weights, for what it's worth, still aren't out as I write this, held past the 14 August launch for safety work (Z.ai, 2026).

What this actually changes, by tier

Small Australian site, no security team: nothing. I'm not building you a checklist out of an incident at a company with its own GPU fleet. You aren't going to pre-stage a 743-billion-parameter model, you shouldn't, and pretending otherwise would be exactly the content-farm advice this site exists to be better than. Your host, your WAF and whoever does your monitoring handle this class of problem.

The one thing worth doing costs nothing. Ask them: when you investigate an incident on my systems, what tooling do you use, and what happens if it declines? "We've never thought about it" is itself an answer.

Mid-tier with a security function: the honest answer is probably neither option above. Purpose-built security products already do payload analysis on hosted infrastructure under contractual scoping a general API doesn't offer. Microsoft Security Copilot, CrowdStrike, SentinelOne and Google SecOps all sit there. That's the boring middle path, and the one most of you will take.

Enrol anyway. Yes, I just spent a section on how leaky these gates are: half a percentage point, an approved practitioner refused, access evaporating for people who'd cleared it. Queue regardless, because it's an hour of paperwork against sitting on the default tier at 3am. Anthropic's CVP is org-scoped rather than portable and excludes zero-data-retention accounts. OpenAI's has that 1 September cutoff. Do it on a quiet Tuesday, then check periodically that your access still exists, because August proved it can vanish without an email first.

Forensic capacity isn't optional here because of Australian law, not best practice. Under section 26WH of the Privacy Act, once you suspect an eligible data breach you must assess it reasonably and expeditiously, taking all reasonable steps to finish inside 30 calendar days (OAIC, 2026). Your tooling isn't a statutory factor, but "our analysis stalled because our vendor's classifier wouldn't look at the evidence" is a sentence you'd rather not send a regulator.

Key takeaways

• Hugging Face named the models that refused: Claude Opus and Fable, in the 27 July technical timeline. The 16 July disclosure had said only "the providers". Classifiers read the request, not the requester, while the attacker was "bound by no usage policy" at all.
• Export controls weren't the cause. Fable 5 returned worldwide on 1 July, Mythos 5 for US organisations, and the intrusion started 9 July.
• The gain was measurable. Replicating the attacker's own decoding recovered roughly four times the initial findings, mostly JWTs and platform tokens, because they could finally run the decode at all.
• Daybreak Blue launched 10 August, four weeks late to help. Anthropic's CVP did exist, but a practitioner already approved for it was refused mid-job on 24 June, and Daybreak Blue moves OpenAI's own advanced cyber completion rate from 1.5% to 2.0%. The 95% figure needs Daybreak Red and a different model.
• The two parties closest to the incident disagree on the fix. Hugging Face says pre-deploy an open model you run yourself. Brockman says give your team a hosted agent, his or a competitor's.
• For a mid-sized org the realistic answer is neither: it's a purpose-built security product doing payload analysis under contractual scoping. Self-hosting answers data egress, not provenance.
• For most readers this changes nothing operationally. The useful move is one question to whoever runs your security, not a GPU purchase.

Only half of this was chosen

There's no villain in the refusal, and that's precisely why it's still true today. The intrusion is a different matter. OpenAI ran that evaluation with cyber refusals reduced and the classifiers that normally block this activity switched off, the containment didn't hold, and a company that had agreed to none of it spent four days under attack and weeks cleaning up. Someone chose all of that. Nobody chose the second part.

Anthropic didn't decide to abandon defenders. No safety team wrote a policy saying incident responders should be refused. A classifier looked at a live command-and-control payload and declined, for the same reason it would have declined if the attacker had sent it. Attacker and defender submit the same artefact. Only one is under time pressure, and only one has to explain a 30-day assessment to a regulator.

The gates being built are a real answer. But a gate is software too, with faults, deadlines and eligibility checks that return the wrong answer at three in the morning.

Hugging Face owned the whole stack, which worked because they're Hugging Face. Most of us don't get that, and the honest advice is smaller than the story: find out, in advance and in writing, what your incident response depends on, and whether any part of it is permitted to say no to you.

Then check that it still says yes. Somebody's approved access disappeared last week without a single email arriving first.

---