Multi-agent teams burn up to 5.1 times more tokens than solo models with almost no accuracy gain

Vals AI ran a straightforward test this month: take two frontier models, GPT-6 Sol and Claude Opus 5.5, and pit solo performance against multi-agent teams on the Vibe Code Bench. The teams burned through 1.8 to 5.1 times more tokens than a single agent working alone. For that premium, buyers got almost nothing back.

Four comparisons, one statistically significant win. That single case, GPT-6 Sol at medium reasoning, moved the needle by only 7.3 points. At maximum reasoning effort, the extra agents did nothing measurable for either model.

The numbers behind the token burn

Anthropic's internal testing on Opus 5.5 showed the same shrinking returns as agent count rose. Jumps from ten to 100 agents barely registered after a full day of runtime. Fable 5.1 fared better on a theorem-proving task with larger teams, yet it still trailed Opus 5.5 overall, and even dipped on a separate knowledge base task when scaled up.

Across the board, bigger agent teams bought speed, not better answers. That distinction prompted OpenAI's Noam Brown to weigh in on what multi-agent setups are actually good for. The pattern held outside Vals AI's own benchmark, making the results harder to dismiss as a one-off.

When more agents make things worse

Scaling from 30 to 100 agents on the knowledge base task made output slightly worse, not better. Paying 1.8x to 5.1x more tokens for one statistically significant win out of four comparisons is a bad trade by any budget standard. If you are pricing out agent orchestration for production work, this is a reason to ask your vendor for the same kind of controlled comparison Vals AI ran.

For researchers, the Lean theorem proving result is worth sitting with: gains appeared above ten agents, yet the swarm still lost to a single Opus 5.5 instance. That is a specific, checkable claim, not a vague trend. For teams exploring AI Agent Courses, the data suggests caution before scaling agent counts.

Why this matters for executives and strategy

Multi-agent setups are often sold as the next step up from a single capable model. The Vals AI data provides a useful gut check. Until someone demonstrates a task where team coordination beats raw model quality at comparable cost, agent teams should be treated as a speed tool at best-not a quality upgrade worth the token bill.

For leaders making build-or-buy decisions, the takeaway is clear: do not accept "more agents" as a proxy for "more capability." Demand controlled comparisons from vendors. The token multiplier is real, and the returns, for now, are not. Professionals evaluating AI for Executives Courses can use these benchmarks to ground procurement conversations in hard numbers rather than architecture hype.