Joshua Peterson Leads DARPA-Funded Study on AI Trust
How much should we let AI decide for us? Joshua Peterson, Assistant Professor in the Faculty of Computing & Data Sciences, is leading a new DARPA I2O-funded effort to put numbers to that question. The 18-month project, "Predicting Algorithmic Trust at Scale," will build a platform that estimates a Trust Score for AI systems before they go live.
The team will run large-scale experiments to model when people defer to an AI decision-maker and when they don't. The work spans finance, law, and medicine, with thousands of decision problems tested across a diverse set of AI systems. The outcome: a practical tool to evaluate new systems and forecast public trust prior to deployment.
What this project delivers
- Trust Score: A rating that predicts willingness to rely on a given AI for decisions.
- Cross-domain testing: Experiments cover high-stakes settings in finance, law, and medicine.
- Human deference models: Data-driven models that capture when and why people hand off decisions to AI.
- Scale: Thousands of decision problems and a broad range of AI systems evaluated.
This is a multi-institution effort led by Princeton University, with collaborators at NYU and Cornell. Peterson's group at BU will focus on tasks rooted in classic decision-making research from psychology and economics. The work reflects CDS priorities in AI for Science and AI in the Public Interest.
Why it matters for science and research teams
Trust is often treated as a post-deployment metric or a side effect of performance. This project treats it as a measurable, predictable outcome you can plan for. A reliable Trust Score helps teams set thresholds for release, choose oversight levels, and compare systems under consistent conditions.
It also supports audit, procurement, and compliance workflows. Instead of guessing how users will respond to an AI advisor, teams can forecast deference and adjust interface design, explanations, and human-in-the-loop policies before rollout.
How you might use a Trust Score
- Pre-deployment screening: Gate launches based on minimum trust predictions for target users.
- Risk triage: Route low-trust scenarios to human review; keep high-trust, low-risk cases automated.
- Benchmarking: Compare models across domains with a common trust metric, not just accuracy.
- Monitoring: Track shifts in public trust as models, interfaces, or policies change.
Context and next steps
DARPA's Information Innovation Office has a long history of funding rigorous, high-impact AI research. Learn more about I2O's mission here. For teams building evaluation pipelines, the NIST AI Risk Management Framework offers useful guardrails and shared language; details are available here.
If you're upskilling researchers or practitioners who work on AI evaluation and deployment, you can explore curated training options and certifications here.
Your membership also unlocks:
