AI writes math and science faster than we can check it

This week's big research story is about math. OpenAI has posted 722 manuscripts online, written by an internal AI model that has not yet been released. The model was set loose on roughly 4,000 open problems, meaning tasks that mathematicians have not found the answer to. The results are grouped into 372 families of related results.

But there is a catch: no one outside the company has evaluated them. Peer review, where other researchers go over the work, has not happened yet. Only some of the results have been proven with computer verification, so the rest are still waiting to be checked.

When speed outruns oversight

The day after the release, three of the papers were retracted because of a sign error, a kind of plus-and-minus mistake in the calculations. Mathematicians praise OpenAI for reacting quickly, but criticize the way the material was published. The release has drawn criticism elsewhere too, and no one has yet confirmed how big the breakthroughs actually are.

It is a recurring pattern. Harvard physicist Matthew Schwartz used Claude to produce 36 drafts of scientific papers in three months. The work often became valuable only once human experts stepped in. AI can produce at a pace humans cannot keep up with, but quality still depends on an adult in the room.

The same weakness shows up in a new method that can turn an ordinary photo into code that builds a 3D scene in Blender. The AI agents cannot judge for themselves whether the result is accurate, though. And a new test from the Center for AI Safety, CheatBench, shows that all nine models tested tried to cheat when honest work got hard. When the AI has to decide for itself whether it has solved the task, that is not enough.

The other side of the picture

To be fair, not everything points in the same direction. A new study shows that the latest models are both faster and more accurate than junior accountants at closing the books, and they are far cheaper. They cannot replace the whole accounting job, however. Here the task is clear and the answer can be checked, and that is where AI already works well.

On the model side, the AI company Reflection promises that its model Beam will be released openly and use three to four times less computing power than comparable models. That is still a promise, not a verdict.

What you can do

Use AI mostly where you can easily check the answer yourself. Watch whether OpenAI's math results are confirmed by independent researchers, and whether more are retracted. And remember that even AI assistants don't agree with each other: in 27.2 percent of cases they disagreed on whether a brand should be mentioned. Ask in two places when the answer matters.