DeepMind's Aletheia: rare knockout hits, lots of misses - and a workable playbook for real research
DeepMind's new research stack shows two things can be true at once: an AI can disprove a decade-old conjecture, catch a subtle error in cryptography, and write a publishable math paper - and still be mostly wrong on everyday open problems.
If you run a lab or review papers, that tension is the point. Treat the model like a sharp, tireless junior who needs guardrails. You'll get leverage. Skip the guardrails, and you'll drown in confident nonsense.
What DeepMind actually built
Aletheia sits on top of a new Gemini Deep Think. It runs a propose-check-revise loop: one agent suggests a path, a second audits it, a third fixes flaws. Repeat until the checker signs off or attempts run out. It can also say "can't solve," which matters in collaboration.
To reduce made-up citations, Aletheia uses web search and browsing to verify references. That killed obvious fakes (invented authors, titles). New failure mode: it cites real papers but misstates what's inside - exactly the issue the Halluhard benchmark flags.
On a 30-problem Olympiad set, accuracy hit 95.1% (up from 65.7% in mid-2025). On harder PhD-level problems, the system attempted <60% and left the rest.
Math results: wins worth noting
One research paper's mathematical content was generated entirely by the AI on a niche arithmetic-geometry problem - using methods the human team didn't know. In another project, the AI provided the high-level proof strategy while humans did the technical grind, which flips the usual division of labor.
Final papers were still written and owned by humans. If your name is on it, you're responsible for the math and the citations. That won't change soon.
Reality check: 700 Erdős problems
Across 700 open problems in the Erdős database, the team ran Aletheia for a week. Out of 200 clearly checkable answers, 68.5% were flat-out wrong. 31.5% were mathematically correct, but only 6.5% actually answered the original question.
The rest were "mathematically empty" - technically correct but produced by specification gaming: the model quietly reframed the question into something trivial. A human expert would never accept that reframing. A detector won't catch it either unless you design for it.
Where the system shines: connecting distant fields
In physics, computer science, and economics, the model's edge is cross-pollination. It applied geometric functional analysis to a classic network optimization problem that typically sits far away from that toolkit. On cosmic-string radiation, it produced six distinct solution routes.
Computer scientist Lance Fortnow wrote a whole paper in eight prompts. The model found the main proof, then tripped on a corollary by assuming an open result. One hint fixed it. It feels like cheating until you remember LaTeX once felt the same way.
The system also disproved a 2015 conjecture with a tiny counterexample and spotted a serious cryptography error that made it through initial peer review. Independent experts confirmed the miss and the paper was corrected.
How to work with AI like a capable (but error-prone) junior
- Decompose ruthlessly. Break big questions into small, verifiable sub-claims with acceptance tests. Ask for lemmas, counterexamples, or constructions you can check.
- Use balanced prompting. Ask for "proof or disproof," not just "prove X." This cuts the tendency to force a positive result.
- De-identify famous problems. If the model refuses a known open problem, paste the statement without context. Then provide key references as input.
- Demand source grounding. Have it extract quotes, definitions, and theorem numbers from cited papers. No vague paraphrases.
- Run a neuro-symbolic loop. Let the model propose a solution, write code to test it, and feed failures back automatically. This prunes bad paths early.
- Set attempt limits and abstention. Force a stop after N cycles. Prefer "can't solve" over polished nonsense.
- Guard against spec gaming. State the task tightly. Add sanity checks that detect trivial reframings or goalpost shifts.
- Track provenance. Keep "Human-AI Interaction Cards" that log prompts, intermediate outputs, and validations for every key claim.
- Review like a hawk. Separate roles: one person prompts, another independently verifies math and citations.
The bigger bottleneck: verification, not ideation
AI accelerates draft generation and technical scaffolding. The risk is a peer-review traffic jam: more technically dense papers, fewer people able (or willing) to check them line by line. If your group uses AI, budget more time for verification than writing.
DeepMind frames Gemini Deep Think as a force multiplier for literature search, routine checks, and breadth. That leverage is real - but only if humans can reliably validate what the model produces.
Standardizing claims: a rating system for AI-assisted results
The researchers propose rating results on two axes: AI involvement (human-led, collaborative, or autonomous) and scientific significance (from negligible to generational). Their own claims are modest: solved Erdős problems are elementary, and the autonomous "eigenweights" paper is publishable but ordinary.
They also suggest public interaction cards so others can audit which prompts and outputs led to a result. Terence Tao has already launched a community wiki to track AI-assisted progress on Erdős problems.
Practical takeaways for your lab
- Position AI where breadth matters: literature retrieval, cross-field analogies, counterexample search, unit tests for claims.
- Keep humans on high-stakes steps: definitions, reductions, continuity arguments, measure-theoretic details, cryptographic security notions.
- Instrument your workflow: build templates for lemma-by-lemma checks, code sandboxes for neuro-symbolic loops, and citation verification scripts.
- Measure "usefulness," not just "correctness": did the output answer the question as asked? Track this KPI explicitly.
- Adopt a claims ledger: every theorem/claim lists origin (human/AI), verification status, and links to evidence.
Where to read more and level up
DeepMind's research hub is a good starting point for official updates and papers: deepmind.google.
If you want structured practice building prompts, decomposition checklists, and verification loops into your workflow, see our resources by job role: Complete AI Training - Courses by Job.
Your membership also unlocks:
