AI speeds up consultation analysis, cuts costs, and frees teams to act
Specialist government-built AI, "Consult," analysed over 50,000 responses to the Independent Water Commission's call for views in about 2 hours for £240, with experts needing just 22 hours to check outputs. This helped accelerate the decision to abolish Ofwat by turning a major bottleneck into a quick, auditable step.
Instead of manually sorting thousands of responses, policy teams worked with ready-made themes and prioritised evidence that mattered for recommendations. Manual reviews of key stakeholders ran alongside the AI pass to ensure perspectives were fully considered.
Accuracy was strong: Consult aligned with expert reviewers in nearly 83% of cases, while two human groups agreed with each other only 55% of the time. On formal evaluation, the tool achieved F1 scores of 0.79 and 0.82 across two reviewer groups, outperforming the human-to-human F1 of 0.74. Earlier in the year, it supported the Scottish government's consultation on non-surgical cosmetics (around 2,000 responses) and was also used on the Digital Inclusion Action Plan (around 800 responses).
Why this matters for government teams
- Removes a persistent blocker: turning weeks of sorting into hours of structured insight.
- Improves consistency across large evidence bases while keeping human judgment where it counts.
- Releases specialist time for decisions, options, and delivery instead of admin.
- Scaled benefit: up to 75,000 staff days saved each year, worth around £20 million.
How to apply this in your department
- Start with high-volume consultations where responses exceed team capacity.
- Define a clear taxonomy of themes upfront with policy leads and analysts.
- Run an AI categorisation pass, then perform targeted human QA on samples and high-salience stakeholders.
- Track agreement metrics (e.g., precision/recall and F1) and document your methodology for transparency.
- Use secure, compliant environments and datasets; log prompts, changes, and reviewer notes.
- Integrate with tools staff already use to speed adoption. For example, Microsoft Copilot trials indicate time savings of around two weeks per official per year.
Consult sits within the "Humphrey" suite. Another tool, "Redbox," supported 5,330 officials at its peak by summarising long documents and drafting notes. As secure enterprise options matured, the team shifted to new AI Exemplars focused on planning decisions, probation support, and other priority services. Redbox has been open-sourced, and its engineers contributed to the upcoming GOV.UK Chat.
Minister's view: "This shows the huge potential for technology and AI to deliver better and more efficient public services for the public and provide better value for the taxpayer. By taking on the basic admin, Consult is giving staff time to focus on what matters - taking action to fix public services. In the process, it could save the taxpayer hundreds of thousands of pounds."
Results at a glance
- 50,000+ responses processed for the water sector review.
- ~2 hours to categorise; ~£240 compute cost; ~22 hours expert checking.
- ~83% agreement with reviewers; human-to-human agreement ~55%.
- F1 scores: 0.79 and 0.82 across two reviewer groups; human-to-human F1: 0.74.
- Projected annual savings: up to 75,000 staff days and ~£20 million.
- Proven on multiple consultations, including Scottish non-surgical cosmetics and the Digital Inclusion Action Plan.
Guardrails that keep quality high
- Use AI for thematic sorting and triage; keep humans in the loop for interpretation and recommendations.
- Always run targeted manual reviews for high-impact stakeholders and outliers.
- Publish methods and metrics so decisions are traceable and defensible.
- Conduct bias checks on themes and outputs; refine taxonomies as you learn.
- Protect sensitive data with approved environments, access controls, and audit trails.
Build team capability
If you're standing up an AI-enabled consultation workflow or training policy teams to use these methods, see practical course options by role at Complete AI Training.
Your membership also unlocks:
