The debate over AI safety just moved from think-tank white papers to employee resignation letters.
Jacob Coxon, a former Anthropic pre-training researcher who previously worked on GPT-4o at OpenAI, quit on Tuesday after three years and posted a blunt indictment on X: 'Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.'
Coxon's departure triggered a cascade of public statements from people still inside the labs.
The numbers being floated are not small
Evan Hubinger, an alignment science lead at Anthropic, reposted Coxon's message and added his own estimate: 'I personally think it is >10% within the next decade.' Geoffrey Hinton, the Nobel Prize-winning computer scientist widely called the 'Godfather of AI,' told the BBC it was 'not unreasonable' to put the same figure on the risk of AI wiping out humanity within a decade.
Marcus Williams, a member of OpenAI's safety oversight team, went further. Asked to quantify the risk on X, he said '70% in the next 3 years if there isn't regulation/slowdown,' while adding that 'regulation/slowdown is very possible.'
The alignment problem has no solution on the table
The through-line in nearly every statement is the same: no one has a credible technical plan. Anna Wang, who works on AGI safety and alignment at Anthropic, wrote on X: 'There is not yet a viable scientific plan to solve risks from recursively self-improving AI.' Tomek Korbak, an AI safety researcher at OpenAI, said plainly: 'Neither Anthropic nor OpenAI are on track to solve alignment to a degree sufficient for shipping superintelligence and we need to slow down.'
Paul Christiano, OpenAI's new safety hire announced this week, put it starkly in a public statement: 'If we build superintelligence without more robust alignment I expect we will permanently lose control of it. If that happens then most people could die.'
Not everyone inside the labs shares the alarm. Chris Hayduk, a life sciences researcher at OpenAI, wrote that 'AI has been and will continue to be an extremely beneficial technology to humanity' and argued 'the conversation should be around how many billions of lives it will save.'
What this means for founders building on top of these platforms
The people shipping the infrastructure your product runs on are publicly divided on whether that infrastructure ends civilization. That is not a framing problem — it is a product-risk and dependency problem that belongs in your board deck.
Free markets run on information, and the information coming out of these labs this week is unusually clear: the frontier is moving faster than the safety work. Founders who are integrating AI into core workflows — payments, health, legal, logistics — should be asking their vendors hard questions about model governance, not just capability benchmarks. The competitive pressure driving these labs is real, but so is the liability exposure if the insiders turn out to be right. Build with that in mind.