Financial Services Firms Face Unresolvable AI Governance Risks
Generative AI presents structural risks that cannot be eliminated, only managed, according to a new report from the London Foundation for Banking and Finance and the Institute and Faculty of Actuaries. The study warns that the financial services industry has yet to develop satisfactory governance answers to the questions the technology raises.
The report, It's Still Not Magic: Framing the Risks Facing Financial Services in the Gen AI Era, surveyed senior practitioners and found that 70% believe AI risks rank among the greatest threats to their sector over the next five years. Three-quarters said those risks have increased substantially since generative AI became widely available.
The top three concerns are cyber threats, misleading outputs, and knowledge gaps among staff.
The Core Problem: Features, Not Flaws
The report identifies a central tension: the same capabilities that make AI valuable-its persuasiveness, accessibility, and ability to operate at scale-also make it difficult to govern, explain, trust, or contain.
As firms embed AI deeper into their tools and infrastructure, many risks shift from individual firms to the entire financial ecosystem. A decision that makes sense for one organisation can create hidden dependencies and shared points of failure across the system.
Keyur Patel, research associate and report author, said the question firms face is not whether risks can be eliminated, but how much risk they will accept in exchange for AI's benefits. "AI outputs can be useful, confident and wrong at the same time-and 'mostly right' can be dangerous," Patel said.
UK Insurance Adoption Outpaces Understanding
AI adoption in UK insurance has reached near-saturation. The Bank of England and Financial Conduct Authority's 2024 survey found that 95% of insurance firms already use AI-the highest rate among all financial services subsectors.
Yet adoption has moved faster than comprehension. Only 34% of firms claim complete understanding of the AI they use, while 46% report only partial understanding.
Concentration of AI infrastructure compounds the risk. A third of all AI use cases in UK financial services now rely on third-party implementations, up from 17% in 2022. The top three providers control the lion's share of cloud, model, and data supply-the exact kind of shared dependency the report identifies as a systemic vulnerability.
The FCA has warned that AI-enabled pricing could benefit many policyholders through tailored premiums while simultaneously making others uninsurable. AI systems used in pricing, policy drafting, and claims handling carry direct regulatory implications under the Consumer Duty. Firms bear the regulatory consequences of misleading AI outputs regardless of whether they built or purchased the model.
Regulators Take a Hands-Off Approach
The UK's AI governance strategy remains principles-based rather than prescriptive. The FCA's 2025-2030 strategy commits to a "tech-positive" outlook focused on outcomes rather than fixed rules, with no AI-specific regulation planned.
FCA chief executive Nikhil Rathi reaffirmed that position in December 2025, citing the technology's rapid evolution every three to six months as a reason to avoid locking down rules. That places the burden of governance squarely on firms.
Political scrutiny is intensifying. The Treasury Committee has been examining whether existing frameworks are adequate. The Critical Third Parties regime, introduced by the Bank of England, PRA, and FCA in November 2024, gives regulators new oversight powers over firms providing critical AI and cloud services, with formal designations expected in 2026.
Insurers with EU clients face additional obligations under the EU AI Act, which classifies AI used in life and health underwriting as high-risk.
What This Means for Your Firm
As AI becomes embedded in underwriting, claims triage, and customer communications, governance infrastructure must keep pace. The report's finding that misleading outputs rank among the top three AI risks is not theoretical. In insurance, a confident but incorrect AI output in a claims decision or policy recommendation carries real regulatory and reputational consequences.
For professionals in AI for Finance and AI for Insurance, the takeaway is clear: understanding these risks is no longer optional.
Your membership also unlocks:
