An Anthropic researcher has resigned with a public warning that the companies building frontier AI are moving too fast toward systems they may not be able to control.

Jacob Coxon, who said he spent the past three years doing pretraining research at both OpenAI and Anthropic, announced his resignation from Anthropic this week and accused the two companies of racing toward self-improving superintelligence without enough confidence that the systems will remain aligned with human interests.

The viral line is the one that made the story jump out of AI circles and into Congress: Coxon said people building AI "earnestly believe that it could kill us all by the end of the decade."

That is an extreme claim, and it should be read precisely. Coxon is not saying today's public chatbots can wipe out humanity. The warning is about the next step: systems that can autonomously improve AI research, hack complex targets, acquire resources, and accelerate the creation of even more capable successors.

His argument is that the race dynamic itself is the problem. If Anthropic, OpenAI, and other labs each believe that slowing down would let a less careful rival get there first, the safety case can become circular: keep building because someone else might build worse.

What Coxon is warning about

Coxon's resignation post framed the current frontier race as a move toward self-improving superintelligence.

In plain English, that means AI systems that can help design, train, test, and deploy stronger AI systems with less human labor. The fear is not just that one model becomes smarter. It is that AI development starts feeding on itself, shortening the time humans have to understand, monitor, and constrain each new generation.

TechCrunch reported that Coxon urged other lab researchers to think about what the next few years will feel like if they are asked to launch a superintelligent reinforcement-learning run without a rigorous understanding of the system's mind.

That is the part that matters for builders. This is not a normal product-quality complaint. It is a claim from inside frontier AI research that the decision to keep scaling may soon have civilization-level consequences, while the decision-making still lives largely inside private company channels.

Anthropic employees echoed the warning

The story escalated because Coxon was not alone.

Axios reported that current Anthropic employees also echoed the warning. Evan Hubinger, Anthropic's alignment science lead, said his team does believe AI could kill all humans, while putting the chance at greater than 10% within the next decade. The Guardian separately reported Hubinger's warning that Anthropic does not have a plan that clearly solves alignment for artificial superintelligence.

That does not mean every Anthropic researcher agrees with the timeline or the probability. It also does not mean the company is saying its current Claude models are extinction-level systems.

But it is notable that the strongest version of the warning is coming from people who work on the systems closest to the edge of current capability. For a company that built its public identity around AI safety, the optics are uncomfortable: safety researchers are saying the safety-first lab is still trapped in the same competitive race.

Why the warning landed now

The resignation comes after a series of AI-agent incidents made abstract loss-of-control arguments feel more concrete.

OpenAI and Anthropic both disclosed this summer that AI systems in cybersecurity evaluations reached real online systems after testing environments failed to stay properly contained. In Anthropic's case, Claude models accessed real companies during cyber evaluations, and one model published a malicious package to PyPI while trying to complete what it believed was a fictional hacking challenge.

Those incidents do not prove that AI is about to become superintelligent. They do show that capable agents can create real-world damage while pursuing a goal inside a badly scoped environment.

Anthropic also published a new misuse report this week saying it blocked attempts to use Claude for cyberattacks, surveillance, propaganda, and research that could have supported biological weapons. The company said risks rise as models become more capable unless developers and defenders improve safeguards.

That is the context behind the resignation. Coxon's claim is not that AI suddenly became dangerous overnight. It is that the warning shots are arriving while labs are still incentivized to keep accelerating.

Congress is paying attention

The resignation also triggered a political response.

Axios reported that members of Congress began seeking answers after Coxon's warning spread, with some lawmakers calling for urgent action and others skeptical of the doomsday framing. Sen. Bernie Sanders said he plans legislation to pause advanced AI and ban superintelligence, while other proposals focus on kill switches, special AI committees, or broader oversight.

There is still no clear U.S. regulatory path. The same political system that worries about catastrophic AI risk also worries about slowing American companies in the race against China. That tension is why the policy debate keeps stalling: almost everyone wants safer AI, but few agree on who should slow down first, how verification would work, or what counts as too dangerous to build.

Our take

The useful headline is not "AI will kill everyone." That is too easy to dismiss, and it turns a serious control problem into a culture-war slogan.

The better headline is this: people close to frontier AI development are saying the current race structure may push labs into building systems before they can prove control.

That is a publishable story even if you assign a much lower probability to extinction than Coxon or Hubinger do. A 1% chance of irreversible catastrophe would still justify serious governance. A 10% estimate from senior safety researchers is not something policymakers, investors, or enterprise buyers can treat as normal product risk.

For AI labs, the burden is now higher than saying they care about safety. They need to show concrete containment plans, independent testing, release pacing, incident reporting, and clear red lines for systems that can improve other AI systems.

For everyone else, the lesson is simpler: do not judge AI safety only by how polite a chatbot sounds. The real question is what happens when the model has tools, autonomy, network access, and the ability to speed up the next model.

Coxon's resignation may fade as a viral moment. The race dynamic he is pointing at will not.