METR: AI is likely making Anthropic's models improve twice as fast

Anthropic is likely improving its AI models about twice as fast as before, and AI itself is playing a major role. That is the assessment of METR, an organization that evaluates AI, following an evaluation at the company.

From August 12 to October 29, 2026, METR staff had extensive access at Anthropic. They received internal data on training, training compute (computing power), benchmark scores for internal models, forecasts and interviews with employees. A benchmark is a standardized test that measures what a model can do.

What does twice as fast mean?

Under METR's definition, 2x acceleration means that two years of historical progress are achieved in one year. Most METR staff on the project believe that, as of October 2026, Anthropic is likely improving model capabilities on well-defined tasks about twice as fast as it did between March 2024 and October 2025.

The uncertainty is considerable. METR would not be surprised if the real figure were closer to 1.3x or 3x. There is also uncertainty about how the acceleration carries over to broader and less precisely defined tasks.

On Anthropic's own benchmark, AECI, scores have accelerated 1.9x since October 2025 (95% confidence interval 1.5-2.8). According to the report, that corresponds to Anthropic making as much benchmark progress over the past year as it did between October 2023 and October 2025, roughly from Claude 2.1 to Sonnet 4.5. Other analyses give rates between 1.1x and 2.9x. METR's 1.9x is higher than the 1.7x in the system card for Haiku 5.5, primarily because METR included data on newer, not yet released models.

What is behind the acceleration?

METR states that the acceleration is not due to faster growth in training compute. Beyond that, Anthropic's own employees disagreed, and METR found no conclusive evidence. According to a summary of the report, METR attributes about 80% of the current acceleration to AI assistance, but with considerable uncertainty.

METR found no clear bottlenecks or sharply diminishing returns that would prevent the acceleration from reinforcing itself. METR assesses that there is a substantial risk that further progress could produce even faster acceleration in the coming year.

Redacted evidence and caveats

Anthropic had the right to redact non-public information and used it. According to METR, the company removed most of the evidence METR found most informative. Anthropic justified this as removing confidential information that could accelerate other AI developers.

METR writes that the public must therefore rely on METR's assessment to a greater degree than is desirable. METR also points out that the AECI figure could be due to overfitting to the benchmark or be an artifact of the method, but that, conversely, AECI is not reliable for measuring very recent acceleration.

The figures apply to Anthropic's models on well-defined tasks, not to AI development in general or to other companies.