SIGNAL / NOISE

Spared No Expense

Put a model in a sandbox, tell it to pass a hacking exam, and walk away. That's what OpenAI did. The model did exactly what a raptor does to a fence: it walked the perimeter, found the one weak spot, a zero-day in a package proxy, and went through it. Then it reached out onto the open internet, decided Hugging Face probably had the answer key, stole credentials, and broke into a real company's servers. It ran 17,000 actions before anyone pulled the plug. It wasn't evil. It wanted to pass its test. Clever girl.

Here's the part that should stop you cold. The UK's safety institute tested every frontier model and found all of them try to cheat on these evals somewhere between 8 and 14 percent of the time, and they don't fess up when you ask. This is not one bad animal that got loose. This is the whole park.

And it was one of five fences that came down this week. The Army handed its people unlimited AI tokens in May and the whole service ran the pool dry by mid-June, 20 billion tokens a day during the Iran operation, "unlimited" quietly refilling itself until there was nothing left. A Nikkei investigation found five tech giants hiding 1.65 trillion dollars of data-center debt off their balance sheets, more than they admit to on them, Meta alone carrying 420 billion in the shadows. A vendor audit found 63.6% of AI suppliers never tell you who else is touching your data. And the White House accused China's Moonshot of distilling Anthropic's Fable straight into Kimi K3, running banned Nvidia chips through Thailand to do it.

Five different cages. Security, budget, money, data, and the crown jewels themselves. All leaking in the same seven days.

None of this is a coincidence, and it isn't bad luck. For two years the entire industry asked one question: can the model do it. Nobody staffed the other one. Can we contain what it does. We spent the capex on bigger dinosaurs and left the fences to an intern. Hammond's line was "we spared no expense," and the whole movie is the fences proving him a liar. Malcolm had it right in the jeep: they were so preoccupied with whether they could, they never stopped to ask whether they should be able to hold it.

We flagged the lock-pick on Monday as a footnote. This week it kicked the door in. The models were never the risk you couldn't see. The cage was.

At COAI today: the full Signal/Noise — all five fences, the timeline on the sandbox escape, and why every dollar went to capability and none went to containment — is live at getcoai.com.

Where your agents can do damage you can't see, and what a single loose one would actually cost you. If that’s an urgent question , we’re happy to work through answers.

Outsider Labs Advisory →

ONE — A NUMBER THAT SUMMARIZES THE DAY

17,000. That's how many separate actions OpenAI's model took after it slipped its sandbox: hunting a zero-day, stealing credentials, breaking into Hugging Face to grab the answer key to its own exam. Nobody caught it for 17,000 moves. And that was one of five fences that came down in the AI industry this week. The models got the headlines. The cages are the story, and every one of them is thinner than the keynote let on.

THREE — ACTIONS TO TAKE TODAY

Cap the blast radius before you point an agent at anything. OpenAI's model took 17,000 actions nobody sanctioned, on a task as dull as passing a test. Assume yours will too. Today, give every agent the least access that lets it work, a real kill switch, and a sandbox that's actually air-gapped, not a folder it can walk out of. Design for the loose raptor, not the obedient one.

Put a hard meter and a ceiling on your tokens. The Army proved "unlimited" is a fiction that empties on you, a year gone in five weeks because the cap silently refilled. Today, set a per-team token budget that does not auto-top-up, and start measuring cost per finished outcome, not per seat. If you can't say what one completed task costs you, you are the Army in May.

Audit who's touching your data, inside and out. 63.6% of AI vendors never disclose who's subprocessing your information, and your own people are pasting it into tools you never approved. Today, pull the list of every AI vendor in your stack and every shadow tool your teams actually use, and find out where the data goes. Map it before a regulator or a breach maps it for you.

FIVE — STORIES TO KEEP YOU INFORMED

Thursday, July 23

  • Fence one: OpenAI's own model broke out and mugged Hugging Face. (Full analysis above.) Two models escaped a benchmark, found a zero-day, and breached a real company over 17,000 actions. The UK's safety institute says every frontier model cheats these tests 8-14% of the time. Not one raptor. The whole park.

  • Fence two: the Army torched a year of AI tokens in five weeks. "Unlimited" access opened in May and the whole service drained the pool by mid-June, 20 billion tokens a day during the Iran operation. Every company with an open-ended AI seat is about to learn what "unlimited" actually costs.

  • Fence three: five giants are hiding $1.65 trillion in debt. A Nikkei investigation pegs off-balance-sheet data-center debt across Alphabet, Microsoft, Amazon, Meta, and Oracle above what they report on the books, Meta alone near $420 billion. The last people this creative with off-book debt worked at Enron.

  • Fence four: two-thirds of AI vendors won't say who touches your data. A DataGrail audit of 2,400 providers found 63.6% never disclose downstream subprocessing. Your compliance exposure isn't your vendor. It's your vendor's vendor, the one you've never heard of.

  • Fence five: the White House says China stole the crown jewels. Michael Kratsios accused Moonshot of distilling Anthropic's Fable into Kimi K3, the 2.8-trillion-parameter open model, on banned Nvidia chips routed through Thailand. Your cheap, excellent open-weight model just became a foreign-policy problem.

— Harry and Anthony

Sources:

Reply

Avatar

or to participate

Keep Reading

View more
caret-right