The People Building AI Are Quitting and Saying It Could Kill Us. Sam Altman Told His Staff OpenAI Would Slow Down, If the Other Labs Do Too
About This Episode
Bloomberg reported on September 10 (relayed by Reuters on September 11) that Sam Altman told a company-wide OpenAI meeting the firm is open to pacing development of its most advanced AI, perhaps in concert with other labs, while conceding some labs may not agree. The remark caps a week in which Anthropic researcher Jacob Coxon resigned saying the labs are 'racing straight to self-improving superintelligence and gambling with our lives' (150M+ views), Anthropic's own alignment lead Evan Hubinger publicly agreed and put extinction odds above 10% within a decade, two more safety researchers left Anthropic and Google DeepMind for the independent evaluator METR saying 'there are no adults in the room', and OpenAI itself asked Congress for mandatory national safety rules. Both companies are heading toward IPOs, and Fortune reports OpenAI began training a model more powerful than Astra on August 28.
Our Take
Everyone with a hand on the wheel now says they would slow down, but only if everyone else does first, and there is no machinery to make that promise binding, so the people who believe it are leaving to build an outside referee while OpenAI asks Congress to force its own hand.
Continue Reading on Unscarcity
Anthropic Pulled the AI Alarm. Who Can Act on It?
Its framework of a capability warning versus an emergency protocol (a pre-negotiated pause mechanism, verification, and liability triggers) is exactly what Altman's 'we'd slow down if the others would' lacks.
AI 2027: The Scenario That Maps the Next 18 Months
Coxon cites the AI Futures Project as his model; its two paths, race or slowdown, branch at precisely the decision Altman described to his staff.