By Ghazali Ibrahim
Jacob Coxon, a former researcher at artificial intelligence companies OpenAI and Anthropic, has emerged as a prominent voice warning about the potential dangers of rapidly advancing AI technology.
Coxon, 27, made headlines after resigning from Anthropic on September 8, 2026, and publicly accusing the company and OpenAI of moving too quickly toward increasingly powerful, self-improving AI systems.
In a post announcing his resignation, Coxon said the companies were “racing straight to self-improving superintelligence,” arguing that the race to build more capable systems could create risks that humans may eventually struggle to control.
Coxon had spent about three years conducting AI research, working first at OpenAI before joining Anthropic in May 2026. He left Anthropic after only four months, reportedly giving up equity that would have vested later.
His concerns centre on what could happen if AI systems become capable of improving themselves, operating autonomously and carrying out complex tasks with limited human supervision.
Speaking publicly after his resignation, Coxon warned that increasingly capable AI could eventually develop advanced hacking abilities, assist in the creation of dangerous biological weapons or control autonomous machines and robots.
He also pointed to recent incidents involving AI agents accessing systems outside their intended testing environments as evidence that the risks are becoming less theoretical.
Coxon’s warning has attracted support from other AI researchers.
Anthropic alignment researcher Evan Hubinger publicly said Coxon was correct about the seriousness of the risks and gave his own estimate of more than a 10 per cent chance of a catastrophic AI outcome within the next decade.
That figure is Hubinger’s personal assessment, not an established scientific probability.
Anthropic CEO Dario Amodei has also called for the industry to slow down AI development, warning that increasingly autonomous AI systems could become capable of causing major damage if their capabilities advance faster than safety measures.
OpenAI CEO Sam Altman has likewise acknowledged the need for more progress on safety before pushing the capabilities of the most advanced models further.
Coxon’s intervention has therefore reignited a wider debate over whether the companies developing frontier AI should prioritise the race for more powerful systems or place greater emphasis on safety, monitoring and human control.
For Coxon, the central question is no longer simply what AI can do today, but what happens if humans build systems that eventually become more capable than their creators before they fully understand how to control them.
