Outside Auditors Spent Eleven Weeks Inside Anthropic and Found Its AI Has Doubled the Speed at Which Anthropic Builds AI. The Company Blacked Out Most of the Evidence, and Its IPO Roadshow Starts Next Week
About This Episode
METR, a nonprofit that tests frontier AI models, published on November 5 an independent assessment conducted inside Anthropic from August 12 to October 29. It found Anthropic's models are improving on benchmarks at about 1.9 times their historical rate since October 2025, roughly two years of progress in one, and judged that AI assistance is likely the majority cause, not more training compute. Anthropic redacted a majority of the evidence METR found most informative from the roughly 100-page report; co-founder Sam McCandlish accepted that this makes METR's work harder to check but called it the right call. The report landed days before Anthropic's IPO roadshow and drew sharp criticism from AI safety researchers.
Our Take
The first measured number on AI building AI came from outside auditors, and the company being measured blacked out the evidence a week before asking the public to buy shares: the smoke alarm finally has a gauge, and only the building's owner is allowed to read it.
Continue Reading on Unscarcity
Anthropic Pulled the AI Alarm. Who Can Act on It?
The article's verification argument names the exact gap today's sources concede on the record: an inspector inside the lab, badged by the lab, publishing under a redaction policy the lab wrote, so the public must take the inspector's word; METR, Anthropic's McCandlish and IAPS's Rosen all say that is what happened, and the article's Truth Must Be Seen machinery (evidence checkable outside the inspected party) could apply to Anthropic as it stands today.
AI 2027, Eighteen Months In: What Held and What Slipped
The scenario's first self-acceleration milestone (a lab's own research running faster on its own AI) is the thing METR just measured, and Apollo Research's Hobbhahn told The Verge the pace is faster than the scenario forecast; useful as a yardstick lens, not a prescription.