Sakana AI model improves itself but shares blind spots across training roles

Sakana AI and UC Berkeley researchers have built a language model that trains itself without a larger model checking its work, achieving a nearly 70 percent win rate against its base model after just two training cycles. The method points toward cheaper self-improving systems but also exposes a structural weakness: when a model grades its own homework, its blind spots get reinforced rather than corrected.

The technique, called Multi-Agent Self-Supervision (MASS), splits one model into three roles. It solves problems, evaluates its own answers, and rewrites the training process for the next round. No outside teacher model intervenes at any stage. The results, published October 12, show gains across all three functions, but the improvements are correlated rather than independent.

How the self-improvement loop works

MASS creates a closed training cycle. The model generates solutions to tasks, then evaluates those solutions using the same underlying capabilities it used to produce them. Finally, it refines its own workflow based on those evaluations and feeds the output back into the next training round. This happens without an external supervisor or a stronger model providing corrective feedback.

The win-rate jump to nearly 70 percent came after two cycles. That is a measurable improvement for a system with no external supervision. It suggests a model can bootstrap its own performance by acting as both student and examiner, provided the tasks align with what it already does well.

Where the blind spots come from

The setup contains an obvious vulnerability. If the model's judgment is flawed in some way, that flaw does not get corrected. It gets amplified. A bad call on what counts as a good answer in round one gets copied directly into round two's training data. The solver, evaluator, and optimizer roles share underlying capabilities, so they improve together rather than functioning as separate checks on each other.

Sakana's own results bear this out. The study found mixed performance on research tasks and blind spots that persisted across training rounds. Win rates climbed against a weak baseline, but the paper offered no proof that the model's overall capability ceiling moved higher. Errors and flawed reasoning can pass through the evaluation process and feed into the next cycle, reinforcing mistakes instead of catching them.

Why this matters for researchers and product teams

MASS is a useful data point for anyone building self-improving systems, but it demands a careful distinction. "Better at beating its past self" is not the same as "more reliable." A model that trains itself on solution traces and then gets better at evaluating and optimizing too tells us these three roles draw from the same well. They cannot serve as independent quality control checks.

For teams building AI agent workflows or pitching self-correcting systems, the finding is practical. Test whether later training rounds widen blind spots or simply make them harder to detect. Evaluations like AstaBench may need to become more adversarial to expose the gap directly. The lesson is to verify reliability gains against external benchmarks, not just internal win rates.