Harvey adds 500 contract tasks to legal AI test

Harvey, which supplies AI tools to lawyers, has expanded its Legal Agent Benchmark (LAB) test with 500 new tasks on contract work. The expansion was announced on June 12, 2026. A benchmark is a standardized test used to compare how well different AI models handle the same type of task.

The 500 tasks are spread across 10 contract types, including commercial agreements, IP/licensing, mergers and acquisitions (M&A), financing, real estate, employment, data/privacy and disputes.

All or nothing

Tasks are graded on an "all-pass" principle. This means the AI agent must make all the relevant revisions and find all the red flags in a contract. There is no partial credit. That is one reason scores on LAB are generally low.

Harvey's post about the expansion contains no results for individual AI models. The figures below therefore do not apply to the 500 contract tasks.

The benchmark currently measures individual negotiation steps. It does not measure autonomous contract negotiation, where the agent negotiates on its own and refers the most important decisions to humans. Harvey mentions this as future work.

Low scores across the full LAB

The full LAB covers 1,749 tasks spread across 24 practice areas plus in-house contract work, graded against more than 100,000 criteria. The tasks, criteria and test environment are available openly on GitHub under an MIT license. Mercor manages the leaderboard together with Harvey.

On the version of Mercor's list that the page showed, Gemini 4 Argon is at the top with 27.4 percent, followed by Grok 4.7x with 19.5 percent and Muse Spark 1.2x with 19.4 percent. Opus 5 has 17.8 percent.

Another test, another picture

Artificial Analysis runs a separate test together with Harvey, LAB-AA v1.1, which was released in October 2026. It uses 120 private tasks spread across 24 practice areas. Version 1.1 checks every deliverable against the task's source documents. Material hallucinations, meaning fabricated information such as wrong party names, amounts or dates, cause the task to fail.

Here Meta's Muse Wonder Full (Max) is at the top with 13.1 percent. Muse Wonder (Max) has 10.0 percent, and Grok 4.7 (Xhigh) has 9.4 percent.

Percentages from the two lists and from different versions cannot be compared directly, and rankings depend on version, date and method.

Agents at scale

On May 5, 2026, Harvey announced that more than 500 agents for specific use cases were live on the platform and that the Agent Builder tool had entered early access. According to Harvey, legal teams have created more than 25,000 custom agents, and more than 100,000 lawyers at 1,500 organizations in more than 60 countries are customers. CEO Winston Weinberg says the legal field has now moved past AI as an assistant and into the age of agents.