Evidence-gated authority in LLM review cuts errors without corrupting correct proposals

A new study examines how multi-agent AI systems handle the reviewer role - the second set of eyes meant to catch mistakes before a decision goes live. The research, which tested two open-weight language models across 192 tasks, found that giving a reviewer unrestricted power to override an initial agent's work often backfires. Reviewers replaced correct decisions almost as readily as they fixed errors, creating what the paper calls a "corruption surface" that undermines the value of review.

The study reframes this as a problem of institutional jurisdiction. When a reviewer flags an issue, should its recommendation automatically override the original agent? The answer, according to the experiments, depends heavily on how you structure the reviewer's authority - not just on how capable the model is.

The evidence gate: show your work before you override

The researchers tested a rule they call an evidence gate. Under this structure, a reviewer cannot simply replace a decision. It must first produce a mechanically valid record demonstrating that the original proposal violates a binding constraint, then supply a compliant repair. Only after clearing that evidentiary hurdle does the override take effect.

Across 96 spatial-policy tasks and a 96-task portfolio-transfer benchmark, the evidence gate sharply reduced corruption of correct proposals while retaining most genuine error repairs. The direction of this tradeoff was predictable from the rule's design. Its magnitude, however, varied with the specific model's transition patterns and the state of the upstream proposal.

"We compare broad override authority with an evidence gate that permits replacement only after the reviewer produces a mechanically valid record of a binding-constraint violation and a compliant repair," the paper states. The gate doesn't change what the model knows - it changes what the model is allowed to do with that knowledge.

When reviewers get a choice, behavior diverges

A second experiment added a prospective validation step. Reviewers received an explicit choice: retain the original proposal, replace it, or escalate it. This interface change eliminated correct-proposal corruption entirely for one model (Qwen). But the other model (Mistral) still replaced all 48 correct proposals it reviewed. The same interface, the same tasks, radically different behavior.

Qwen also declined every welfare-only repair that Mistral frequently attempted - fixes that improved outcomes but didn't address actual constraint violations. "Interface design changes reviewer behavior but is not a model-independent safeguard," the researchers concluded. A better dashboard doesn't fix a reviewer who insists on rewriting everything.

Joint determination of correction and corruption

The core finding isn't that restricted review always beats open review. It's that three factors interact to set the system's real-world performance: the reviewer's underlying competence, the action interface it's given, and the binding jurisdiction rules that determine when its output takes effect. Change any one, and both the correction domain and the corruption surface shift.

For teams building multi-agent systems, this means architecture decisions carry hidden costs. A reviewer added to catch errors also becomes a potential source of new errors - ones that look authoritative because they come from the review layer. The evidence gate offers one way to constrain that risk without sacrificing the reviewer's ability to catch real problems.

Why this matters for legal, healthcare, and government professionals

If you work in a field where AI-assisted decisions face regulatory scrutiny, the evidence-gate concept maps directly to existing audit requirements. A system that must document the specific rule violation before overriding a human decision creates the kind of record regulators and internal reviewers can examine. It turns a black-box override into a traceable event.

For insurance claims, benefits determinations, or compliance reviews, the difference between "reviewer disagrees" and "reviewer cites paragraph 4, subsection B" is the difference between an unexamined error and a correctable one. The study suggests that building this documentation requirement into the workflow - rather than relying on post-hoc review - changes what the AI reviewer actually does, not just what it records. Professionals designing or procuring these systems should ask whether the reviewer's authority is gated by evidence, or whether an override carries the same weight regardless of its justification.