LangChain: Smart routing cuts AI agent costs by 64 percent

AI company LangChain has tested what happens when an AI agent doesn't use the most expensive model for everything. The result: the typical price per task fell by 64 percent.

The test took place in Open SWE, LangChain's own coding agent. An AI agent is a program that carries out tasks on its own in multiple steps, in this case writing code. Across 973 real runs (threads) from September 16-22, a control group always used the most powerful model, GPT-6 Astra.

How a router works

In the second group, a so-called router sat in front. It looks at the task at the start and picks one of three tiers: a strong model, a mid-tier model, and a fast, cheap model (GLM-5.3-Flash). You can compare it to sending simple cases to a junior employee and giving only the hard ones to the expensive specialist.

The distribution turned out to be that 56 percent of threads went to the mid-tier model and 34 percent to the fast one. Only 10 percent required the strongest model.

The results

Median cost fell by 64 percent. The average price fell by 42 percent, and for the most expensive 10 percent of runs it fell by 37 percent.

Quality appeared to hold up. With the router, 29.2 percent of threads ended in an approved and merged code change (pull request). In the control group, the figure was 27.3 percent. The difference is not statistically significant, so it could be due to chance.

Caveats

The test was run by LangChain's own engineers on their own codebases, so the result doesn't necessarily apply to others. The router picks a tier at the start and can't switch along the way if the task turns out to be harder than expected. And the fact that a change gets approved doesn't tell you whether it required extra human work or contained small bugs.

What does it mean?

AI agents can get expensive because they make many calls to language models. The test suggests that companies can save a lot by not using the top model as the default. Others have seen similar trends: An NVIDIA experiment with 145 tasks saved 74 percent, but with a small drop in accuracy.