A team of data scientists has built a framework that forces AI coding agents to keep a structured, evolving memory of every experiment they run. The system, called Arbor, delivered 2.5 times the performance gains of standard agents on real engineering tasks while spending the same budget.
The researchers come from the Gaoling School of Artificial Intelligence at Renmin University of China and Microsoft Research. Their core argument: long execution alone does not produce better results. Without a way to remember what failed and why, agents burn tokens repeating the same dead ends. Arbor addresses that by treating the research process itself as a tree that grows, prunes, and refines over time.
How the hypothesis tree works
Arbor is built on three requirements. It must branch to test competing hypotheses without letting that branching collapse into chaos-what the researchers call "branching with coherence." It must separate short-horizon execution tasks from the long-lived strategy that decides which direction to pursue next. And it must distinguish real, verified improvement from overfitting to a particular trial.
Persistence sits at the center of the design. The tree links hypotheses and ideas, the code or configuration artifacts used to test them, experimental evidence like results and metrics, and distilled insights. One example: "this data filter helped, but this learning rate scheduler didn't."
A long-lived coordinator manages the overall research strategy, updating nodes, selecting promising leaves, pruning or merging branches, and propagating reusable lessons across the tree. Short-lived executors spin up isolated worktrees to test single hypotheses. As results return, the tree narrows and refines. The researchers describe it as "the operational research state of the system"-simultaneously the search frontier, the memory of past attempts, and the audit trail for verified artifact improvement.
Testing against memoryless agents
The team tested Arbor in an autonomous optimization setting. Agents received an initial research artifact-a data pipeline, harness, or training script-and had to improve its held-out performance through iterative experimentation, with no human steering. Held-out performance measures how well models generalize to data they have not seen before.
Tasks spanned model training, harness engineering, and data synthesis. Across these, Arbor outperformed the average held-out gains of Codex and Claude Code by 2.5 times for the same resource budget. The researchers said the takeaway is straightforward: keeping a structured, evolving hypothesis tree yields more improvement than running the same models as memoryless coding agents.
What the industry sees
Mahmoud Ramin, a research director at Info-Tech Research Group, said Arbor's design mirrors how people actually work. "Arbor accumulates information over time and allows agents to build upon prior discoveries just as humans do, through learning, adaptation, and eventually building upon what they have learned in the past."
He pointed to the tree's ability to retain relevant data from prior attempts as its most innovative feature. But he also raised a concern about auditability as these systems scale. "As autonomous agents become more capable of performing work without human operators overseeing them, enterprises will need transparency into how and/or why an agent took a specific action or reached a certain conclusion."
Why this matters for IT and development teams
For teams already using coding agents, the 2.5x performance difference under the same budget changes the math on what is worth building internally. The gain does not come from a better model-it comes from architecture that preserves state across sessions. That means the same budget spent on a tree-based framework like Arbor produces measurably better results than running a memoryless agent longer.
Professionals exploring this space can find deeper training on the underlying techniques through AI Research Courses or targeted material in AI for IT & Development. The shift toward persistent research state also raises the auditability question Ramin flagged. Teams adopting autonomous research loops will need to plan for logging and transparency early, not as an afterthought.
Your membership also unlocks:
