AI Could Kill Us: Anthropic Researcher Resigns with Dire Warning
Jacob Coxon, a researcher who previously worked on pretraining at both OpenAI and Anthropic, resigned from Anthropic on Tuesday, issuing a stark warning that the companies are “racing straight to self-improving super-intelligence and gambling with our lives.” His departure highlights escalating concerns within the AI industry regarding the rapid development of advanced systems without adequate safety protocols.
Coxon’s public statement on X revealed a chilling internal sentiment: “The people building AI earnestly believe that it could kill us all by the end of the decade.” This dire prediction, echoing themes from classic science fiction, was surprisingly affirmed by other senior Anthropic staff, underscoring a profound internal conflict between technological advancement and existential risk.
A Resignation Driven by Existential Fears
Coxon, who spent three years in pretraining research, explicitly accused both Anthropic and his former employer, OpenAI, of irresponsible development practices. He emphasized that the industry’s pursuit of self-improving superintelligence is occurring despite a widespread, private belief among researchers that such AI systems pose a significant threat to humanity’s survival.
Internal Affirmation of Catastrophic Risk
Coxon’s warnings were not isolated. Evan Hubinger, Anthropic’s alignment lead, publicly corroborated the sentiment, stating, “We really do earnestly believe AI could kill all humans.” Hubinger personally estimated the odds of this catastrophic outcome to be greater than 10% within the next decade. Samuel Marks, who leads Anthropic’s scalable oversight division, further explained that researchers continue development due to financial incentives and the fear that others might develop the technology first, potentially leading to its misuse. Marks noted that many AI developer staff “desperately want to slow down to figure out how to build AI more safely.”
The Unchecked Race for Superintelligence
The rapid pace of AI development, described by Coxon as a “race,” is a central point of contention. Despite internal warnings and calls for a slowdown—including from Anthropic itself since June, and OpenAI following its GPT-6 Astra debut—companies appear to be pushing ahead. Hubinger admitted that Anthropic “does not yet have a plan to solve alignment for superintelligence and are not clearly on track to” develop one. This lack of a clear safety roadmap for increasingly sophisticated AI models fuels the concerns of researchers like Coxon.
Broader Industry and Political Context
The internal dissent at Anthropic reflects a broader tonal shift within the AI industry, where initial enthusiasm for transformative potential is increasingly tempered by warnings of uncontrolled systems. OpenAI’s chief scientist, Jakub Pachocki, also recently cautioned that “No one is prepared for the consequences of a continued rapid rise in machine intelligence.” However, this push for caution faces external pressures, such as the Trump administration’s argument that slowing down development risks ceding the AI arms race to competitors like China. This complex interplay of internal fears, corporate incentives, and geopolitical competition underscores the profound challenges in governing the future of artificial intelligence.
Sources & Reference Data
CoinDesk, Variety, Bbc, Theverge
PS: For educational purposes only. Not financial advice. Investing involves risk.