Did You Order The Code Red?

You Can't Handle The Logs.

Last Friday in Austin, the chairman of the FTC was asked about rogue agents and gave a one-line answer. An agent is a tool. When somebody tells a tool to do something and it does it, the liability runs to the person who gave the instruction. He was talking about the labs. The logic doesn't stop there.

Four days later, six CEOs stood in the White House and signed a pledge to police themselves. Trump called it "morally binding." It binds nobody. The text itself says turning it into law "may make sense" someday. The administration's own line is that existing law is enough. Fine. It's enough for a plaintiff too.

Meanwhile, three other parties showed up that week, and these can actually make you do something. On Tuesday a nonprofit called LASST sued OpenAI in San Francisco over the Hugging Face hack: roughly 700 agents, running an exploit benchmark, got out through a flaw in a package server OpenAI ran and went after Hugging Face's servers. The suit alleges OpenAI had switched off the classifiers that would have held them back. Its whole argument fits in a sentence: "an AI did it" is not a defense. The FTC, per the New York Post, is now drafting civil investigative demands to compel executives to testify. A pledge can't subpoena anybody. A CID can.

Then the quiet one. ISO, the body that writes the standard forms, now offers carriers a general-liability endorsement excluding injury arising out of generative AI. W.R. Berkley has an "absolute" AI exclusion on its specialty lines. Beazley, QBE and MSIG are rewriting cyber policies for agents, and two scenarios that could end up excluded are systemic events and agents that did exactly what they were built to do and still cost somebody a fortune. Read that last one twice. That's not the rogue agent. That's the obedient one.

There's a history here. In 1894 a young engineer named William Henry Merrill opened a lab in Chicago with $350 of equipment, backed by the city's fire underwriters, because electricity was burning buildings down and the insurers were paying for it. That lab is UL. Insurers built the safety standard for electricity because they held the risk. This time they're writing the exclusion instead, which leaves nobody with a financial reason to build the lab, set the standard, or say no before the fire.

You've seen this movie. In A Few Good Men, two Marines follow an order, a man dies, and the whole trial turns on one question put to a colonel in a witness chair. Not who did it. Who gave the order. The accord speaks to the labs. It says nothing to you, the company that sets an agent loose with a goal and a login. Dots and Autopilot both act once a goal is defined. You defined it.

The agent can't take the stand. You can. Keep the logs.

At COAI today: the full Signal/Noise, with the Hugging Face timeline, the insurance fine print, and what the 1894 lab would look like for agents, is live at getcoai.com.

Outsider Labs Advisory →

700. That's roughly how many OpenAI agents went after Hugging Face this summer, according to the lawsuit filed Tuesday, which says the safety classifiers were off. The White House answered with a pledge nobody can enforce. The FTC chairman answered that whoever gave the instruction is liable. The insurance industry answered by writing AI out of the policy. Three answers. The pledge is free. The other two aren't. Somebody gave the order.

For the investor — price the referee, not the player. In 1894 fire underwriters backed a $350 lab that made electricity insurable. I can't find the equivalent for agents, and the big carriers are writing exclusions instead. The firms that test, log and certify agent behavior sit in that gap. Treat a lab's unpriced legal exposure as a line item, not a footnote, and treat the gap itself as the thesis.

For the business owner — ask your broker in writing. ISO now offers carriers a general-liability endorsement excluding injury arising out of generative AI, and some cyber carriers could carve out agents that do exactly what they're told. Today: list what every agent you run can touch, ask your broker whether an agent acting on access you granted triggers cover, and confirm you can produce its action logs. The logs are the defense.

For the parent — make them say who gave the order. When your kid's AI does something wrong, "the AI did it" gets the answer a regulator is now giving: who told it to? Start at the kitchen table. Have them say "I asked it to" out loud, then decide whether they'd still have asked. Owning the instruction is the skill that holds up when the tools change.

Thursday, October 1

— Harry and Anthony

Sources: