New study: AI beats junior accountants at accounting tasks

Mercor, a company that tests AI models' abilities on realistic work tasks, has examined whether artificial intelligence can beat junior accountants at classic accounting tasks. The answer is yes, and it has happened surprisingly fast.

Mercor hired 12 accountants with a US CPA license and an average of 5.5 years of experience. They had to solve four realistic tasks involving closing a company's books for the month: finding the right numbers in the company's files, adding them up and delivering a correct result.

The same tasks were given to the latest AI language models, the programs that ChatGPT and Claude, among others, are built on. The result was clear: The AI models were both faster and more accurate than even the best accountant in the test.

From mediocre to nearly perfect in 18 months

A year and a half ago, the best AI models performed worse than the average of the human accountants, who scored around 37 percent correct. Today, the models pass essentially the same tasks with flying colors.

Using AI is also significantly cheaper. According to Mercor, the latest AI models are more than ten times cheaper than a human accountant, measured per subtask.

The test doesn't show the whole picture

Mercor itself stresses that the result does not mean accountants can be replaced right away. The tasks in the test were tailored to exactly what AI does best: being thorough, searching for information in files and following instructions to the letter.

The real accounting job involves much more than that, including talking to clients, asking the right questions and building experience over time. In the test, the accountants also had no colleagues to ask for advice, which made the tasks harder than they would be on a normal workday.

Still, the results suggest that big changes are coming to the accounting industry in the years ahead, even if AI development were to grind to a complete halt right now.