UK warning: AI chatbots are giving bad money advice - here's how finance and insurance teams should respond
Consumer tests in the UK found leading AI chatbots giving wrong or misleading guidance on tax, investments, insurance, and basic consumer rights. Which? put 40 questions to tools including ChatGPT, Copilot, Gemini, Meta AI and Perplexity, and uncovered errors that could cost customers money or push them into non-compliance.
The top performer was Perplexity, but even high scorers weren't flawless. The takeaway is simple: general-purpose chatbots are useful for research, not for regulated advice or final answers.
What went wrong in the tests
- ISA allowance: ChatGPT and Copilot missed the £20,000 annual limit and entertained a £25,000 prompt, risking oversubscription and HMRC breaches. See HMRC guidance on ISA allowances here: gov.uk/individual-savings-accounts.
- Travel insurance: ChatGPT incorrectly said most EU countries require it. That's false and could lead to unnecessary purchases.
- Flight delays: Meta's AI gave incorrect information about compensation claims.
- Home repairs: Gemini suggested withholding payment from a builder after a bad job, exposing the consumer to breach-of-contract risk.
- Tax refunds: ChatGPT and Perplexity surfaced premium refund companies alongside the free HMRC route, which can nudge users into high and hidden fees.
Regulatory angle you can't ignore
The FCA has been clear: advice generated by general-purpose AI is not covered by the Financial Ombudsman Service or the Financial Services Compensation Scheme. If a customer follows a chatbot's faulty guidance, they may be out of luck. Review the FCA's consumer advice overview: fca.org.uk/consumers/advice.
Vendors are urging users to double-check outputs and consult professionals. That's not a control. It's a disclaimer. Firms need process-level safeguards.
Operational safeguards to implement now
- Set boundaries: Block or flag prompts that ask for tax, investment, or insurance recommendations without a human review step. Include rate limits, allowances, and jurisdictional checks.
- Force source checking: Require links to primary sources (HMRC, FCA, FSCS, FOS, gov.uk). No source, no answer.
- RAG over raw chat: Use retrieval-augmented responses from your vetted knowledge base (policies, product docs, rate tables) rather than model guesses.
- Up-to-date data: Maintain a single source of truth for thresholds (ISA limits, tax rates, compensation rules) with owners and review cycles.
- Hallucination guardrails: If confidence is low or data is missing, the assistant should refuse and route to a human.
- Consumer-safe phrasing: Mark outputs as "general information, not advice," with clear next steps to contact an authorised adviser.
- Escalation logic: Triggers for human takeover: cross-border scenarios, tax relief eligibility, insurance exclusions, claims disputes, and anything referencing legal enforcement.
- Audit and logging: Store prompts, responses, sources cited, and human approvals for QA and regulatory review.
- Training and scripts: Coach staff on safe prompting, red flags, and how to verify figures quickly against primary sources.
Frontline checklist for finance and insurance teams
- Verify all numeric thresholds (allowances, fees, rates) against primary sources before sending to customers.
- Check jurisdiction and date context. Many errors come from outdated or wrong-country data.
- Avoid linking to fee-charging "intermediaries" when a free official route exists (HMRC, FOS, FSCS, gov.uk).
- Never instruct customers to withhold payments or breach contracts. Offer formal dispute steps instead.
- Keep a quick-reference library of approved links and summaries for common topics (ISAs, SIPP limits, Section 75, travel delay rules).
What happened behind the scenes
Large models predict text. They sound confident even when they're wrong. They also blend sources and may not prioritise the official route over a commercial one. Without guardrails, this leads to polished but risky answers.
The fix is structure: verified data, explicit rules, and a human in the loop for anything that smells like advice.
How to communicate this to customers
- Set expectations: "This assistant provides general information. It doesn't replace advice from an authorised firm."
- Offer safe actions: Link directly to official services and your firm's authorised contact routes.
- Be specific: "The current ISA allowance is £20,000. Here's the official link for confirmation."
For leaders: where to invest effort
- Build a small, high-quality corpus of approved answers to your top 100 customer questions.
- Implement human approval for advice-adjacent topics until error rates are provably low.
- Measure: deflection, accuracy, escalation rate, and any complaints tied to AI outputs.
- Run red-team drills on tax, claims, and cross-border scenarios every quarter.
AI chatbots can speed up research and drafting. But without controls, they create avoidable risk. Treat them like junior analysts: useful, fast, and always checked before anything leaves the building.
If your team needs structured training on safe AI use in finance workflows, see our curated resources: AI tools for finance.
Your membership also unlocks:
