Agentic AI Systems Match Human Economists on Causal Analysis Tasks
A new study comparing agentic AI systems to human economists on identical causal-inference work finds the AI produces median estimates comparable to human performance, though humans show wider variation in their results. The research also tests whether AI models can reliably review and rank research submissions-and finds they produce consistent rankings across different reviewer models.
The experiment had two parts. First, researchers ran replicated causal-inference tasks with multiple AI systems and human economists, measuring how much their estimates varied. Second, they created a review tournament where three different AI models (Gemini 3.1 Pro Preview, Opus 4.6, and GPT-5.4) each evaluated the same 300 submission groups and ranked them.
What the rankings showed
The AI reviewer models produced consistent rankings across all three models. Codex GPT-5.4 ranked first, Codex GPT-5.3-Codex second, Claude Code Opus 4.6 third, and human researchers fourth. The consistency of these rankings across different reviewer models suggests AI can reliably evaluate research artifacts.
Human economists produced wider-tailed distributions of estimates than the AI systems-meaning humans occasionally made more extreme judgments rather than performing systematically worse overall. On the median, AI and human performance aligned.
Why this matters for research teams
Two practical implications stand out for people building empirical workflows. Agentic AI can generate point estimates and analysis code that match human performance on median, which removes a significant bottleneck in empirical work. AI reviewers can also perform consistent cross-model evaluation, which enables automated screening, ranking, and reproducibility checks at scale.
This opens concrete applications: automated literature screening, pre-analysis code review, and synthetic replication work that researchers currently do manually.
What to keep in mind
Variance within individual AI model instances was substantial, so per-instance performance matters for deployment decisions. The study's external validity depends on the chosen tasks, prompt protocols, and reviewer models-results may not generalize to other econometric problems or research domains.
The researchers flag reproducibility across additional tasks, larger model pools, and open benchmarks as key questions to watch. Prompt engineering, chain-of-thought control, and calibration methods all affect per-instance variance and hallucination rates, areas where further testing is needed.
For practitioners working on empirical research pipelines, this work demonstrates that agentic AI can perform at parity with human economists on causal tasks and reliably evaluate submissions. The practical path forward involves testing these systems on your own domain-specific tasks and monitoring instance-level variance in deployment.
Learn more about AI Research Courses and AI Data Analysis Courses to build skills in this area.
Your membership also unlocks:
