Anthropic wants government to be able to stop dangerous AI models

Anthropic, the company behind the AI assistant Claude, wants the US Congress to give authorities the power to halt AI models that could cause catastrophic harm. The proposal appears in an essay by the company's CEO, Dario Amodei, titled "Policy on the AI Exponential." The essay was published in June 2026, and sources give the date as either June 10 or 11. It is accompanied by draft legislation that Anthropic is presenting to Congress.

From transparency to binding rules

The proposal goes further than the current approach, in which AI companies are primarily expected to be open about their models. According to Implicator.ai, Amodei draws a comparison with aviation: the government should be able to block or revoke a model's rollout if it fails safety tests. Anthropic cites biological threats and major cyber vulnerabilities as examples of catastrophic harm.

As part of its argument, Anthropic points out that the company's own model, Claude Mythos Preview, found thousands of serious vulnerabilities in operating systems and browsers. According to IBTimes UK, the company believes AI companies should not bear sole responsibility for deciding whether their models are safe enough to release.

Who and what is affected

The rules would apply only to the largest models. That means models trained with more than 10^25 floating-point operations (a measure of computing power) and/or developed by companies with more than $500 million in AI revenue or more than $1 billion in AI research and development spending. The sources are unclear on whether the criteria are cumulative or alternative. According to Implicator.ai, the thresholds would capture around five major labs, including OpenAI, Anthropic, Google and Meta.

Independent third parties would test the models in four areas: cybersecurity, biological weapons, loss of control over AI systems, and automated AI research and development. Developers would also have to publish safety information and robustly secure model weights and training infrastructure. Model weights are the numbers that make up the "brain" of the trained model itself.

The proposal also includes civil fines tied to a company's global annual revenue that increase for repeated violations.

Jobs, medicine and criticism

The essay also touches on the labor market. It proposes measures to address job displacement, including wage insurance, tax incentives, an expanded social safety net and possible universal capital accounts. Amodei also asks the US Food and Drug Administration (FDA) to speed up approval of AI-designed drugs.

Implicator.ai reports that Anthropic will provide "significant financial support" for the legislation, and that the company's lobbying spending in the US has risen from $360,000 to $1.6 million year over year.

David Sacks, the administration's former AI chief, is critical. He calls the initiative "a sophisticated regulatory capture strategy based on fear-mongering." That means he believes the company is using fear to shape the rules in its own interests. The criticism comes from one of the sources and has not been checked against a primary source.