Astronomer adapts ray tracing to flag AI uncertainty at scale
An astronomer at the University of Arizona has introduced a method that helps AI models recognize when their outputs shouldn't be trusted - even for systems with billions to trillions of parameters. The work is posted as a preprint on arXiv, with code released for public use, and received support through the National Science Foundation's EAGER program (Early-concept Grants for Exploratory Research).
At the center is Peter Behroozi, associate professor at Steward Observatory. His goal is straightforward: reduce "wrong-but-confident" predictions - the kind of hallucinations that lead to false citations, flawed medical calls, or biased decisions at scale.
The core idea: ray tracing for billion-dimensional models
The method adapts ray tracing - the technique used to model how light moves through media - to explore the high-dimensional spaces where neural networks learn. A student's question about how light bends through Earth's atmosphere sparked the approach. As Behroozi put it, "Instead of doing this in three dimensions, I figured out how to make it work for a billion dimensions."
This matters because traditional tools to probe uncertainty break down as models grow. In his galaxy formation work (including the Universe Machine), existing methods couldn't map parameter uncertainty well enough to match the scale and nuance of modern datasets.
Bringing Bayesian sampling back - without the prohibitive cost
The technique operationalizes Bayesian sampling for large models. Rather than trusting a single network, it trains thousands of siblings on the same data to capture a distribution of plausible answers. The spread becomes a direct signal of uncertainty.
That approach has been a gold standard for smaller systems, but it's typically far too slow for modern AI. Behroozi's method delivers orders-of-magnitude speedups, making it practical to quantify uncertainty in models that previously ran blind. As he summarizes it: ask a panel of experts, not just one; when they disagree, you proceed with caution.
Why this is useful for science and industry
High-stakes decisions in medicine, finance, housing, energy, criminal justice, and autonomous systems all suffer when models overstate their confidence. A model that "knows when it doesn't know" can trigger second opinions, additional testing, or human review - the same way a cautious clinician orders a follow-up rather than rushing treatment.
For researchers, this directly addresses a trust gap. Whether you're inferring black hole properties, screening drug candidates, forecasting weather, or summarizing literature, uncertainty that's visible and calibrated reduces the need for expensive post-hoc validation and builds confidence in AI-assisted results.
Practical takeaways for your lab or team
- Quantify epistemic uncertainty with ensembles/Bayesian sampling; set thresholds that route high-variance cases to human review or additional data collection.
- Log uncertainty alongside every prediction (variance, predictive intervals). Avoid single-number answers when stakes are high.
- Use disagreement across sampled models for out-of-distribution detection; integrate this signal into your safety gates.
- Start with a subsystem to benchmark cost and calibration (ECE, NLL, Brier score). Scale where uncertainty improves outcomes, not just metrics.
- Fetch the released code via the arXiv preprint's resources and test it on your current model before rearchitecting pipelines.
What's next
The paper is awaiting peer review, and the code is public. Beyond safer AI, the technique unlocks an ambitious scientific goal: recovering the initial conditions of our own universe, moving from simulations that merely "look right" to reconstructions grounded in data-driven uncertainty.
If you're skilling up your team on uncertainty-aware AI, Bayesian methods, and production deployment, browse curated options at Complete AI Training.
Your membership also unlocks:
