Anthropic Spent the Year Saying AI Labs Can't Check Their Own Homework. Then It Quietly Handed Congress the Answer Key: a Brake That Fires Only Above a Billion Dollars or Fifty Bodies, Pulled Only by a Judge
About This Episode
POLITICO reported on October 16 that Anthropic representatives have privately pitched lawmakers a confidential draft federal bill on frontier AI safety. The draft would let the government block a model's release or use only for a foreseeable, imminent risk of more than $1 billion in damages or more than 50 deaths or serious injuries, from a bioweapon, a model escaping its developer's controls, or a cyberattack; it would run oversight through lab-paid evaluators registered with Commerce, require a federal judge to order any suspension, bar other agencies from acting, and preempt some state AI laws. Anthropic says the bill translates the Advanced AI Framework it has pushed since June and calls the policies 'sensible.'
Our Take
The lab that spent 2026 saying AI companies cannot check their own homework has quietly written the federal brake it wants applied to itself, and the question for a listener is who gets to set a brake's height when the driver is the one holding the pen: the trigger fires only above a billion dollars or fifty bodies, the inspectors are paid by the inspected, one agency and one judge hold the only switch, and the state rules that might have reached lower are overridden.
Continue Reading on Unscarcity
Anthropic Pulled the AI Alarm. Who Can Act on It?
The article argues a capability warning is theater without pre-negotiated machinery (a defined trigger, verification that is not chosen and badged by the lab, liability) and that brakes must be designed before the designer knows whether it will be the one constrained; today Anthropic itself drafted the trigger, set it above every incident so far, and kept the evaluators on the lab's payroll.
Who Investigates the Machine? An NTSB for AI Agents
Its core test for any oversight regime is that the operator must not scope, pay or schedule its own investigator, the lesson aviation drew when it cut the NTSB loose from the Commerce Department in 1974; the draft bill has labs paying their evaluators and makes Commerce the single gatekeeper.