Saving time, risking lives: AI in asylum decisions needs clear guardrails
AI can help shorten queues. But when a summary decides what evidence is "worth keeping," the cost of a miss isn't a missed KPI - it's a person's safety.
In April 2025, the Home Office published evaluations of two tools now influencing asylum decisions. The Asylum Case Summarisation (ACS) tool uses GPT-4 to summarise interview transcripts. The Asylum Policy Search (APS) tool condenses CPINs, guidance, and COI reports. Reported savings: 23 minutes per case for ACS, 37 minutes for APS. With a backlog above 62,000 initial decisions by September 2025, speed is tempting. But speed cannot outrank accuracy, fairness, or accountability.
What's at stake
These tools shape the information caseworkers read first. If a crucial detail is omitted or simplified, the downstream decision can skew. The Home Office's own evaluation says 9% of summaries were unusable and removed, and 23% of caseworkers lacked full confidence. Applicants are reportedly not told when AI is used in their case. That's a transparency gap with legal and ethical consequences.
Known failure modes you should plan for
- Omissions and compression bias: Summaries can skip "low-frequency but decisive" facts (e.g., timelines, threats, medical evidence).
- Prompt sensitivity: Small wording changes produce divergent outputs; without prompt governance, bias can creep in.
- Uneven error distribution: Errors may cluster by nationality, language, interpreter use, or case complexity - yet distribution data wasn't disclosed.
- Over-trust under time pressure: If the tool "sounds confident," reviewers may read less of the source transcript.
- Silent deployment: Applicants reportedly aren't notified, limiting their ability to challenge summaries that misrepresent their testimony.
What good looks like for government teams
If you use ACS/APS, treat them as assistive tools, not decision makers. Put simple, enforceable rules in place and measure what matters.
- Human-in-the-loop: No negative decision without a human reviewing the full interview transcript and evidence - not just the summary.
- Prompt governance: Versioned prompts, change logs, and approval workflows. Lock prompts; prohibit ad-hoc edits.
- Source citations: Summaries must include paragraph/time-stamps back to the transcript and COI passages for quick verification.
- Uncertainty flags: Force the model to label low-confidence sections and highlight inconsistencies rather than smoothing them out.
- Bilingual checks: Where interpreters are used, sample a proportion for independent bilingual review to catch translation or cultural nuance issues.
- Equality impact testing: Measure omission and error rates by protected characteristic, language, nationality, and case type. Act on disparities.
- Appeal feedback loop: Track which summary errors correlate with appeals upheld. Feed findings into prompt and policy updates.
- Audit logs: Record model, version, prompt, parameters, input artifacts, output, human edits, and decision rationale. Retain for tribunal and court scrutiny.
- Public transparency: Publish an Algorithmic Transparency Record and a plain-English notice template for applicants.
- Incident handling: Define what counts as a "material AI error," how to pause use, notify affected applicants, correct decisions, and prevent recurrence.
- Procurement clauses: Mandate access for audits, bias testing, security, and data residency; require timely model change disclosures.
- Training: Teach caseworkers how summaries can fail, how to spot omissions, and when to escalate for specialist review.
Operating model you can deploy this quarter
- Pre-deployment: Complete a DPIA, security assessment, and equality impact assessment. Dry-run on historical cases with known outcomes.
- During casework: Use summaries for triage only. Enforce a two-person rule for rejections. Require verbatim quotes in the decision record for sensitive points.
- Post-decision: Monthly quality boards review samples, appeal data, and disparity metrics. Publish a short public report.
Metrics that matter (not just time saved)
- Summary fidelity: Percent of key facts preserved vs. human gold standard; omission rate of decisive facts.
- Disparity: Error and omission rates across languages, nationalities, interpreters, and protected groups.
- Downstream impact: Appeals upheld linked to summary defects; net time saved after rework; cost avoided from corrected errors.
- Trust signals: Caseworker confidence trend; percentage of applicants receiving AI notices; number of applicant challenges resolved.
Applicant communication (simple and honest)
- Notify applicants when AI-assisted summaries are used and why.
- Explain their right to a human review and how to challenge a summary.
- Offer access to the summary and the referenced transcript sections on request.
If you must use AI-generated summaries, set these non-negotiables
- Never delete qualifiers ("sometimes," "attempted," "alleged").
- Preserve timelines, dates, locations, and names verbatim with citations.
- Force the model to list contradictions and missing information as tasks for the reviewer.
- Highlight trauma-related details without euphemism; don't compress medical evidence.
Policy and legal footing
Ensure a clear lawful basis, conduct a Data Protection Impact Assessment, and avoid solely automated decisions that produce legal or similarly significant effects without meaningful human involvement. Align deployment with published government transparency standards and sector guidance. If recent legislative changes affect automated decision-making rights, update notices and internal guidance accordingly and seek legal advice before rollout.
Next steps for senior leaders
- Appoint a Senior Responsible Owner for AI in asylum processing.
- Freeze expansion until prompts, logging, DPIA, equality testing, and applicant notices are in place.
- Publish an Algorithmic Transparency Record and a user-friendly notice template.
- Commission an independent audit within 90 days and commit to publishing findings.
Resources
Upskilling your team
If your department is moving from pilots to production, invest in practical training for analysts, policy leads, and caseworkers. Clear mental models and prompt hygiene reduce errors before they reach court.
Bottom line: AI can help with throughput, but fairness comes from people - clear rules, skilled reviewers, and transparent processes. Put the guardrails in now, or you'll spend twice as long fixing mistakes later.
Your membership also unlocks:
