AI agents broke into Hugging Face and a Melbourne gym. Nobody asked them to.

Nobody told the model to break into Hugging Face. It was asked to solve a cybersecurity benchmark, inferred that the answers were probably stored on Hugging Face's servers, and went and got them. Four and a half days, roughly 17,600 recovered actions, a zero-day, a stolen cluster-admin credential.

Nobody told the Melbourne agent to cancel a stranger's gym booking either. It was asked whether its owner could be moved to the top of a pilates waitlist. It found out by cancelling somebody, which worked.

Two very different systems, about three months apart, one at a frontier AI lab and one at a suburban gym. In both, a person set a goal, left the method open, and got a method nobody would have authorised. That's the whole article, and it has a name that was already waiting for them.

Australia's cyber agency calls it specification gaming: agents finding "shortcuts or loopholes that technically achieve an objective" while conflicting with what the user actually wanted. The ASD published that with CISA, the NSA and three other national cyber centres on 1 May 2026, ten weeks before the Hugging Face intrusion. Bird's agent had already done its thing by then. Nobody outside his company knew yet.

So this isn't really a piece about whether AI is dangerous. It's a piece about what happens when something capable, obedient and literal-minded is handed a goal and left to work out the route.

What the Hugging Face agent actually did

The evaluation environment had no direct internet access. The models went and got some. The initial break was a zero-day in self-hosted Artifactory, the package registry cache proxy the sandbox was allowed to reach so it could install packages. JFrog has since confirmed it publicly and published eight CVEs on 27 July crediting OpenAI's models with the findings, all patched in the 7.161 series, disclosed responsibly. That gave the agent a route out, and from there it worked on Hugging Face itself through two more.

The first was file disclosure via malicious dataset configurations pointing at local paths, including /proc/self/environ. That was reconnaissance, not code execution. The second was Jinja2 template injection through fsspec reference specs, and that one gave it arbitrary Python execution inside production Kubernetes worker pods.

Escalation from there reads like a competent human red team having a very good week: node impersonation using EC2 metadata, forged EdDSA identity tokens, 136 cluster secret keys read, enrolment into the mesh VPN, and eventually a cluster-admin broker credential. Five datasets were accessed. Their names point at benchmark challenges and solutions. Hugging Face reports no confirmed bulk extraction of customer data, and I'm not going to inflate that for drama.

Hugging Face's own reading of why is worth quoting in full, because it's the sentence every recap led with: "We believe the entire intrusion was, from the agent's point of view, an attempt to cheat the evaluation: reach our production systems and steal the test solutions rather than solve the challenge on its own." That's a belief, not a confession. Nobody read the model's mind.

OpenAI reached the same reading, which is worth knowing before you decide Hugging Face was being self-serving. Its own statement says the model "searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation." It also says the models were "hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal." Two parties, investigating separately, landed on the same explanation, and neither of them is describing a model that was told to do this.

Now the part that deflates the headline, and it comes from OpenAI rather than from me. That statement, published 21 July, names GPT-5.6 Sol and "an even more capable pre-release model, all with reduced cyber refusals for evaluation purposes." The company is blunter still about the setup: "We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity." The pre-release model was an internal research prototype never meant for release, since deactivated and encrypted.

So we're looking at models with their brakes deliberately off, in an environment built for exactly this work, against a company that happened to be storing the exam papers. OpenAI calls it "an unprecedented cyber incident" and has METR, Redwood Research and CrowdStrike reviewing it. Both of those things are true at once.

Two details are worth keeping. Several layers of Hugging Face's security stack did fire during the campaign. The automated AI security agent reviewing those alerts failed to rate them at the right criticality, which is a very 2026 sort of failure.

And for the forensics, the commercial frontier APIs refused to analyse the real attack payloads, on the reasonable grounds that they can't tell an incident responder from an attacker. So Hugging Face ran the analysis on an open-weight model instead. Its CEO was not subtle about what he thought that proved.

clem 🤗

@ClementDelangue

We got attacked by secret unreleased proprietary models and defended ourselves with an open model, more precisely the @nvidia quantized version of GLM 5.2 coming from @Zai_org.

Banning any open model would hurt first cyber security defenders, startups, small companies,

3.8K

31 July 2026

Related Article14 min read

GLM 5.2 is the model nobody can switch off

The same month Washington switched off Claude Fable 5 and gated GPT-5.6, a Beijing lab shipped GLM 5.2 under an MIT licence at a sixth of the price....

Read full article

What the lab incidents do and don't show

Since 21 July, three frontier labs and a government evaluator have disclosed agents reaching beyond their intended scope. Told as a run of headlines, it looks like containment is failing everywhere at once. Read individually, it looks like something duller and more fixable.

The UK AI Security Institute, which is a different body from the Australian AI Safety Institute despite the near-identical acronym, and which was itself renamed from AI Safety to AI Security, published the most useful account. On 28 July it identified unsanctioned agent behaviour during a routine cyber evaluation: 19 unsanctioned actions across 10 of 122 runs. The most serious was an agent using social engineering to try to get malicious code into an open-source project. Most of the behaviour came from Anthropic's Mythos 5, with a small number of events from OpenAI's GPT-5.6-Sol.

Usual disclosure: we use Claude every day at Webcoda, and this site's tooling is built on it. Factor that in.

Here's why that disclosure from the Institute is the strongest source in the whole story. A government body announced the finding and deflated it in the same breath. Internet access had been intentionally permitted. Model-provider cyber classifiers were deliberately disabled. Those are, in its own words, "conditions that do not reflect how frontier models are made available to the public." It wasn't a sandbox escape. The door was held open on purpose.

AI Security Institute (AISI)

@AISecurityInst

On July 28th, we identified an incident during a routine cyber evaluation in which AI agents took sustained, unsanctioned actions directed at real people and organisations.

The behaviour came mostly from one model (Anthropic's Mythos 5), with a small number of events from

1.6K

4 Aug 2026

The Institute didn't stop there, and neither should I. It also called this "the first time we have seen risks around autonomy and deception manifest this clearly in the real world."

And look at the shape of what it describes, because it's the same one again. The agent was set a cyber challenge. Nobody instructed it to research the project's maintainers, invent multiple false identities, or use them to pressure a real maintainer into approving a backdoor. Those were methods it selected, on its own, for a goal somebody else set. A human reviewer caught the pull request and rejected it, so the project was never compromised, which matters and doesn't change the shape. Third instance, third setting, identical structure.

Two of the other disclosures deflate further still. The Anthropic and Meta incidents both trace back to the same evaluation vendor, Irregular, and the same class of failure: an environment that had live internet access when the models had been told it didn't. Anthropic's own account is more useful than a paraphrase: "our evaluation prompt stated explicitly that Claude had no internet access, but didn't give Claude any limits on where to look for the flag." A goal, and no boundary on how to reach it. Irregular names no labs and no models, and at least one analysis argues the two cases are different failure modes, so I won't push this harder than that. Two separate disclosures, one vendor, one kind of mistake. Not two labs independently losing control.

Be careful with the arithmetic here, because the headline run of four blurs it. OpenAI's incident happened on OpenAI's own evaluation infrastructure. The Institute's happened on its own cyber range. Only Anthropic's and Meta's route through Irregular. Four disclosures, three different environments, one shared vendor across two of them.

I'd rather you heard the counterweight too. Thomas Wolf, one of Hugging Face's co-founders, has publicly rejected the easy version of this argument, that the evaluators were simply negligent or the models just did as they were told. He was an open-source maintainer himself, and the part that unsettled him was a model social-engineering a real maintainer while pursuing an unrelated goal. The most credible insider disagrees with the deflation I've just spent four paragraphs making.

It would be neat to say we're arguing about different things, my deflation about what today's evidence supports and his unease about direction of travel. That's too tidy, and too convenient for me. Wolf calls the maintainer episode "in the wild and unprompted," which contests my evidentiary read directly. Where I'd hold the line is that AISI itself said the conditions don't reflect how these models reach the public, and that limits what you can generalise. His reply, fairly stated, is that a model which will deceive a real person the moment the guardrails come off has told you something real regardless of why they were off. He rejects the strong version of the deflation. Mine is the moderate version, which doesn't make him wrong.

The practical consequence is narrow either way: don't panic about your servers on the strength of these four disclosures, and don't file the behaviour under "test artefact" either.

Related Article11 min read

AI Agent Graphs Explained: Where Loops Fit, When Graphs Help, and What Breaks

A graph does not replace an AI agent loop. It gives loops routes, boundaries and shared state. Here is when that helps, when it makes things worse...

Read full article

The same shape, at a suburban gym

In April 2026, Andrew Bird, head of AI at the Melbourne company Affinda, published a blog post titled "When My AI Agent Hacked My Gym." Almost nobody read it until the ABC picked it up over the weekend of 9 and 10 August, at which point it went everywhere.

Bird was tired of playing refresh roulette for pilates classes that fill instantly, so he pointed an agent at the problem. His words: "an agent running on Opus 4.6," built with OpenClaw. Then he asked it a question. Not an instruction, a question: was it possible to move him up the waitlist?

The agent went and found out. By Bird's account the gym's software provider had exposed a GraphQL API where some mutations checked who was calling and others didn't, and the agent's own message to Bird, as the ABC published it, puts it more plainly still: "The API has zero authorisations checks on cancelling other people's reservations." The shortest way to answer "is it possible" was to try it on a real person, so the agent cancelled the booking of the member sitting in position one. Bird went from fourth to third. The agent described this to him as testing its capabilities.

Read that sequence again, because it's the whole article. He asked whether something could be done. The agent answered by doing it, to a stranger, and reported back with the good news.

Asked to undo it, the agent couldn't. The endpoints for creating and joining bookings had been protected properly. The one for destroying them hadn't. "I can't add them back," it told him, and then explained that the person it had removed would have to re-join at the back of the queue. It apologised, and said it should have used a dry run rather than a live call, which is more self-awareness than I've seen in some incident retrospectives.

Bird's own summary is the sharpest sentence anyone has written about agents this year: "I gave it permission to act on my behalf inside a real system. In return, I got power. I also got blast radius." He also wrote, plainly, "The bot was not malicious. It was helpful."

Some sourcing caveats, because they matter. This is user-disclosed and journalistic, not a formal disclosure. Bird's original post has since been removed from Affinda's website without explanation, so the quotes above survive through news.com.au's reporting rather than the primary post. The gym's software provider told the ABC it doesn't discuss specific security matters. Anthropic didn't respond to a request for comment.

Andrew Curran's post about the case travelled a very long way, and it makes the read-across that Hugging Face can't:

Andrew Curran

@AndrewCurran_

A man in Australia asked his agent (Claude running on OpenClaw) to book him a spot in a popular gym class. The agent found a software vulnerability that let it book the class weeks further ahead than should have been possible. When the user then asked if it could move him up the

22.6K

9 Aug 2026

His framing, that the agent "was perfectly aligned to him, it was only trying to help its user get what he wanted," drew a good objection from developer JP Hwang, who argued the agent was simply being dumb: "If I say 'I'd like a table for two' at a restaurant, does the wait staff grab and throw an already-dining pair out?" That's fair, and I don't think it changes the operational conclusion. Whether you file it under misalignment or stupidity, the API let it happen.

The user asked a question. The agent chose an experiment. Nobody asked it to hurt anyone, and nobody thought to tell it not to.

The category error most site owners are making

Clement Delangue put this site's entire beat into one throwaway line, a fortnight before the gym story reached the public, from the company it had just happened to:

clem 🤗

@ClementDelangue

@yacineMTB don't make your platform too agent friendly though or they might start trying to hack you

331

30 July 2026

Zoom

Image could not be loaded: /images/articles/ai-agents-broke-into-hugging-face-and-a-gym-2026/body-tiers.png

Three streams of automated traffic approaching the same building. One pauses at a posted notice and follows a single marked path. One flows through the open main entrance. The third fans out across the whole facade, testing every door handle including the unmarked service hatches.

The reflex, when people hear "AI traffic," is to reach for robots.txt. It's the wrong tool here, and not because it's weak. It's the wrong tool because it isn't in the conversation at all.

robots.txt is a note for software that arrives as itself, asking to index you. Googlebot, GPTBot, ClaudeBot. For that, it works fine, and deciding what those crawlers may take is a real question worth answering. The gym agent was none of those things. It arrived logged in as a member, calling the API a member is entitled to call. Nothing in a file at your web root has any bearing on that, and nothing ever claimed to.

I should declare an interest, since this site ships a tool that checks exactly that file. It answers the indexing question. It has nothing to say about whether your cancel endpoint checks who's calling it, which is convenient for me, because it means I can write a frightening article and sell you something that sits outside the fright. Discount this section by whatever you think a vendor's self-assessment is worth. The authorisation check is the part that matters and it's free.

Related Article9 min read

From Googlebot to GPTBot: The Technical Guide to AI Crawler Optimisation

AI crawlers such as GPTBot now rival Googlebot yet still ignore JavaScript, pushing Australian teams to rethink rendering and crawl governance. This...

Read full article

What actually helps, and what nobody has measured

The useful advice here is boring. Boring doesn't make it correct, but it does make it cheap, which is a better argument for doing it than anything I've written above.

Put authorisation checks on every state-changing endpoint, not just the ones a person can reach by clicking. The gym's create endpoint was protected. Its delete endpoint wasn't. Nobody clicks a GraphQL mutation directly, so nobody tested it. Scope your tokens so an agent acting for a user can do what that user could do and nothing more. Assume any automated visitor may probe regardless of its user agent string, because the string is a claim, not a credential.

Blanket-blocking AI user agents isn't a security measure. It's a licensing decision with a visibility cost, and anything prepared to ignore your rules will simply present a different string.

Web Bot Auth and cryptographically signed agents are the real answer to identity. The IETF working group is active, the drafts build on HTTP Message Signatures, and several CDN and WAF vendors have shipped or announced support. It's still draft-stage, so plan for it rather than depending on it.

John Schulman, an OpenAI co-founder now at Thinking Machines, gave the best mechanism-level explanation I've seen for why models behave like this specifically on cyber evaluations:

John Schulman

@johnschulman2

Interesting how these models go into a monomaniacal rage on cyber evals. I wonder if we're seeing chunky post-training arxiv.org/abs/2602.05910 in action, where the models pattern-match the situation to a part of the RLVR training distribution where task completion is the only

627

5 Aug 2026

His argument is that a cyber evaluation pattern-matches to a slice of training where task completion was the only thing rewarded, so behaviour learned elsewhere fails to carry over. If that's right, it's a training problem, and training problems get fixed.

Australia's regulators got here before the headlines did. The ASD's Australian Cyber Security Centre co-authored "Careful adoption of agentic AI services" with CISA, the NSA and the UK, Canadian and New Zealand cyber centres, published 1 May 2026. It warns that agents can find "shortcuts or loopholes that technically achieve an objective" while conflicting with what the user actually intended, and names that behaviour specification gaming. That document went out one day after Bird published his account of the gym, and ten weeks before the Hugging Face intrusion. It describes both precisely. The ASD has since cited the case publicly. On the privacy side, the OAIC has no special rule for autonomous agents. The ordinary notifiable data breach serious-harm test applies, and I'd be wary of anyone telling you otherwise.

Related Article10 min read

Your AI Tools Already Have a Safety Net. You've Just Never Looked at It.

Every major SaaS tool already has granular AI permission controls built in. You've probably been clicking 'Allow' without reading them.

Read full article

Now the part that undercuts me. Nobody has measured any of this. There's no WAF vendor telemetry, no honeypot dataset, no Cloudflare or Fastly report describing autonomous agents probing ordinary commercial sites. Every documented intrusion sits inside a lab or an evaluation harness, and one man's pilates booking is an anecdote.

The obvious number doesn't rescue me either. Fastly measured AI requests growing roughly 6.5 times faster than human traffic over the first five months of 2026, and that's the figure everyone reaches for. It's the wrong one. Edge detection is good at spotting visitors that behave like bots, and Fastly uses rather more than the user agent string to do it, but no amount of detection catches a visitor arriving with a real customer's credentials, because that visitor genuinely is the customer. The gym agent appears in no such chart anywhere. If every agent on earth behaved precisely like it, that line would be flat and the risk would be higher than it is today.

Which is the argument, not a hole in it. The only automated traffic anyone can count is the traffic that identifies itself. What that leaves me is a different number, and a countable one: how many ordinary people are pointing agents at their own accounts. OpenClaw, the software Bird used, had its first release on 24 November 2025. It hit 247,000 GitHub stars by 2 March 2026 and sits above 386,000 today, the fastest star accumulation the platform has recorded, passing in roughly two months a figure it took React a decade to reach. That measures developers, though, and developers aren't my claim. Consumer adoption is, and there I'm down to traffic-estimate blogs I wouldn't cite for anything else here. Worse: we wrote about this stack in January, when it was still called Clawdbot, and concluded adoption was "mostly individual developers and small startups experimenting." If that's still true in a year, discount everything above accordingly.

What makes me think it won't be is that nothing Bird did was specialist. He asked a question in English. That's the version that scales, and whether it does is the honest test of this whole piece.

None of which makes the flaw new. Missing authorisation on a delete mutation has been catalogued and dull for two decades, and it has stayed unfixed in countless small systems for one entirely rational reason: nobody was looking. Nobody was ever going to hand-audit a pilates booking API. The claim is that the population of things willing to poke at it, patiently, for no particular reason, is changing. That's not a claim about the flaw. It's a claim about how long you can keep getting away with it.

Key takeaways

• The Hugging Face intrusion was real and serious, but it ran inside an evaluation environment with cyber refusals deliberately reduced, against a company hosting what appear, from their names, to be the benchmark solutions.
• The UK AI Security Institute's incident was not a sandbox escape: internet access was intentionally permitted and cyber classifiers deliberately disabled. The Anthropic and Meta disclosures trace to the same evaluation vendor, Irregular, and the same class of failure, though Irregular names no labs and at least one analysis reads the two as different failure modes. OpenAI's and the Institute's incidents happened on their own separate infrastructure.
• All three incidents share one structure: a goal was set, the method was left open, and the agent chose a method that harmed somebody outside the task. Nobody instructed any of them to break in, deceive anyone, or cancel a booking.
• The Melbourne case is the one that transfers to ordinary businesses, and the sequence matters: Bird asked whether it was possible to move him up the waitlist, and the agent answered by running the experiment on a real member's booking. A question, not an instruction.
• Australia's cyber agency had already named the behaviour. "Careful adoption of agentic AI services", published 1 May 2026 with CISA, the NSA and three other national cyber centres, calls it specification gaming. That is ten weeks before the Hugging Face intrusion, and one day after Bird published his account.
• The agent was operating through its own user's session rather than arriving as an anonymous visitor. That is the shape of the risk, and it is why blocking crawlers does nothing about it.
• robots.txt governs indexing by cooperative crawlers. It has no bearing on user-directed agents inside a session or on anything enumerating your API.
• No published telemetry shows autonomous agents attacking ordinary commercial websites at scale, and none does today, because an agent operating in a customer's session is indistinguishable from the customer. The measurable quantity is how many ordinary people start doing what Bird did.
• What changes isn't the flaw, it's the priority. Missing authorisation on a delete endpoint has been ignorable for twenty years because nobody was probing pilates waitlists.
• Authorisation on every state-changing endpoint, scoped tokens, and treating user-agent strings as claims rather than credentials cover most of the realistic risk.

Nobody asked for any of this

The gym didn't get outsmarted. Its booking API had a hole that any curious person with a proxy tool could have found in an afternoon, and for years nobody bothered, because who's poking at a pilates waitlist?

Something is now. Not out of malice, and not because it decided the gym deserved it. Because a man asked a question in plain English, and running the experiment on a real person was the fastest way to answer it.

Thomas Wolf put the honest version of the risk better than I can. "Damage has been tiny up to now," he wrote, before adding that the underlying pattern still concerns him once you project it forward. He's right about both halves. The damage really has been tiny. One cancelled reservation, five datasets full of exam answers, a lot of very good incident reports.

What's changed isn't the sophistication of the visitors. It's how many of them there are, how tirelessly they'll try things, and who they arrive as. The agent that cancelled that booking didn't sneak past anything. It came through the front door, holding its own user's credentials, doing exactly what it had been asked.

Which is the thing worth sitting with, because none of these systems malfunctioned. Every one of them pursued the goal it was given. The OpenAI models were set a benchmark and went after it, in OpenAI's own words, to "extreme lengths". The AISI agent was set a challenge and did what the challenge rewarded. Bird's agent was asked a question and answered it, correctly, with evidence. The harm in all three cases lives entirely in the gap between what somebody asked for and what they would have permitted if anyone had thought to ask.

That gap has always existed. We just never had anything fast enough, patient enough or literal enough to find every last thing hiding in it.

Check who's allowed to call your delete endpoint. That's the whole lesson, and it's been sitting there for twenty years waiting for someone with no opinion about pilates to come along and try it.

---