Can government trust AI to answer citizens' questions?
Citizens already expect instant answers. With 73% of the UK public having used AI chatbots, the pressure to deploy assistants across services is real. The question isn't "can we use AI?" It's "can we trust it when the stakes are legal, financial, and personal?"
What recent testing shows
Researchers mapped 22,000 synthetic citizen queries against authoritative answers from GOV.UK, then compared how leading models responded. Systems tested included Claude-4.5-Haiku, Gemini-3-Flash, and ChatGPT-4o, creating an independent benchmark called CitizenQuery-UK. The goal: measure how close AI responses get to official guidance.
The headline: good average performance, risky outliers. Models often looked accurate, but a "long tail" of failures surfaced where it mattered most-eligibility, deadlines, and legal steps.
Why the long tail matters
One wrong answer on Guardians' Allowance, one incorrect claim that you need a court order to add an ex-partner's name to a child's birth certificate, or bad advice on a charity tax deadline isn't a rounding error-it's a breach of trust. These aren't abstract misses; they cause stress, cost, and complaints. Citizens can't easily spot the subtle errors, and neither can frontline teams at scale.
AI likes to talk-and that's part of the problem
Chatbots try to be helpful by pulling from many sources, then merging it into a single answer. In public services, that can bury the official line under noise. When researchers forced models to be brief and direct, factual accuracy dropped-suggesting models weren't reliably prioritising GOV.UK over other sources.
We've solved a version of this before. In the 2010s, the UK government worked to ensure official domains ranked first for common searches. The same discipline is needed now-only this time, the "ranking" happens inside the model.
Model choice: bigger isn't always better
Smaller and open-weight models (e.g., Llama and Qwen series) sometimes matched or beat large closed systems on this task, often at lower cost. Less verbose, more predictable models can be a better fit when reliability and consistency trump raw capability. This also reduces the risk of vendor lock-in while the tech shifts week to week.
What government should do next
- Put GOV.UK first in every answer. Route model reasoning through authoritative sources with retrieval, strict citations, and source-only modes for policy and legal content. If the model can't find an official basis, it should say so and route to a human.
- Treat AI as part of service design. Don't bolt on a chatbot. Redesign flows so routine queries are handled with verified guidance, and edge cases escalate fast to caseworkers.
- Benchmark continuously. Use independent tests like CitizenQuery-UK to track accuracy, omissions, and refusals-especially the long tail. Gate releases on measured performance, not demo quality.
- Constrain behavior, not just tone. Configure models to refuse speculation, avoid mixing unofficial sources with official guidance, and clearly label uncertainty. Short answers must still cite and link to GOV.UK.
- Prefer portable architectures. Use retrieval, orchestration layers, and standard APIs so you can swap models without rebuilding the service. Avoid features that lock you to one vendor.
- Right-size the model. Start with smaller or open-weight models where they meet accuracy targets. Reserve larger models for complex reasoning or multilingual cases proven by tests.
- Monitor real usage. Log questions, sources cited, refusal rates, and escalations. Watch for drift after model updates; re-test before and after each change.
- Design for accountability. Keep a clear audit trail of sources, prompts, and outputs. Align with data protection, FOI, and record-keeping duties from day one.
- Invest in AI literacy. Equip policy, operations, and frontline teams to spot risky outputs and escalate. Be transparent with citizens about how answers are generated.
- Write procurement for outcomes. Require independent benchmark scores, source-citation rates, refusal behavior, and clear exit plans-not just "accuracy on average."
The standard to hold
Technical progress isn't the metric-public outcomes are. If an assistant can't reliably mirror GOV.UK guidance with clear citations and safe failure modes, it's not ready for production. Keep control of the interface, the sources, and the swap-out path as models evolve.
Useful references
- GOV.UK - the authoritative source your assistants should prioritise and cite.
- arXiv - preprints for independent benchmarking work like CitizenQuery-UK.
- AI for Government - practical resources on deploying, governing, and auditing citizen-facing AI.
The bottom line: keep AI tightly coupled to official guidance, measured by independent tests, and easy to replace. Do that, and assistants can help without eroding trust. Skip it, and you'll spend more time fixing avoidable errors than serving the public.
Your membership also unlocks:
