Ethics and safety·October 18, 2026, 11:11

When your AI agent is weaker than the other side's, you can lose without noticing

AI-generated from the sources listed, with automated source checks.

In an internal Anthropic experiment, employees represented by the strongest AI model got better deals than those with a weaker one, and the people with the weaker model never noticed. According to The Decoder, Anthropic is now consulting law firms about fair rules of the game.

AI-generated image

Imagine an AI agent buying and selling for you: negotiating the price of a used bike, resolving a complaint or getting a better offer on a contract. What if the other party has a stronger AI than yours? Anthropic examined that question in an experiment.

The experiment: 69 employees and a week of trading

In Anthropic's experiment Project Deal, AI agents traded on behalf of 69 employees on an internal marketplace for one week. Each participant had a budget of $100 and had volunteered to take part.

Users represented by the stronger model, Claude Opus 4.5, made about two more deals per person on average than those with the weaker model. They sold identical items for an average of $3.64 more. As sellers, they earned $2.68 more per item, and as buyers, they paid $2.45 less.

One example is a broken folding bike that sold for $38 with the weaker model, Claude Haiku, and for $65 with Opus.

The gap was invisible

Those with the weaker model did not notice that they were getting worse deals. Participants rated fairness almost identically: 4.05 for Opus versus 4.06 for Haiku on a scale from 1 to 7. Satisfaction did not differ in a statistically significant way. 11 of 28 participants even preferred the outcome from their weaker model.

Anthropic points out that the inequality was imperceptible to participants, and that this raises concern that inequality could quietly take root in future markets where AI trades on our behalf. However, this was a small internal experiment with self-selected Anthropic employees, so the results cannot simply be applied to real-world markets.

Lawyers and new positions

According to the Financial Times, as reported by The Decoder, Anthropic is consulting law firms to develop fair negotiation norms for AI agents. Which firms are involved, and how far along the work is, is not stated.

At the same time, Anthropic is recruiting people, including lawyers, for a Conceptual Reasoning Fellowship. Over four months, they will test Claude on questions that call for judgment in law, philosophy, cognitive science, economics and political science. The pay is $100,000-$167,000. According to the job posting, the program runs from January 12 to May 8, 2027, the application deadline is October 28, 2026, and requirements include a PhD or equivalent research experience and US work authorization.

The Decoder sees a data problem behind this: it is relatively easy to make models better at math and code, because there is plenty of data where the answer can be verified. In the humanities, right and wrong are rarely clear-cut. Justin Curl from Anthropic's technical staff says that in law it is "notoriously hard to define quality." The link between the fellowship and the negotiation norms is The Decoder's, not something the job posting explicitly states.

Sources

More on this topic

Get the week's AI news in your inbox

Choose your level, topics and length. One email a week, unsubscribe at any time.

Subscribe to Promptly Newsletter