Who's Deciding Your Fate? Florida Lawmakers Target AI's 'Black Box' In Insurance Claims
Florida's insurance regulator has a clear message: AI can help, but it can't be the final say-especially on denials. Insurance Commissioner Michael Yaworsky told senators the state needs oversight to keep "black box" decisions out of claims handling.
The urgency is real. In a recent filing, a health insurer couldn't explain its own model's logic to regulators. "We don't know" won't cut it with lawmakers-or with consumers who get denied.
What the bills would do
Rep. Hillary Cassel and Sen. Jennifer Bradley have filed identical bills (HB 527 and SB 202). The core requirement: if a claim is denied, a human must be involved in that decision, not just an algorithm running behind the scenes.
The intent is simple-no silent, automated denials. If an AI system informs a decision, there needs to be human accountability and a clear explanation path.
Industry pushback-and the caution flag
Some industry voices argue existing laws already cover unfair or deceptive practices regardless of who-or what-made the decision. That point will get tested as AI systems become more embedded in claims workflows.
House Speaker Daniel Perez has designated the second week of December as "Artificial Intelligence Week" for the House. His warning: short-term shortcuts can create long-term consumer risk. Expect hearings, definitions, and tighter expectations.
What this means for insurers
If you use AI in FNOL, triage, SIU referral, medical review, or claims settlement, assume you'll need a verifiable human-in-the-loop for adverse decisions. Also assume you'll be asked to explain the decision path in plain English.
Black-box vendors won't shield you. If you can't explain why a claim was denied, you'll carry the regulatory burden.
Practical steps to get ahead
- Map decisions: List every claim decision point where AI assists or auto-acts. Flag all denial or partial denial touchpoints.
- Insert human review: Require adjuster sign-off for any decline, rescission, or adverse payment determination influenced by a model.
- Build explainability: Keep model cards, feature lists, and reason codes that can be shared with regulators and consumers.
- Audit trails: Log inputs, model versions, thresholds, overrides, and human approvals. Keep retention policies aligned with exam cycles.
- Adverse action notices: Make notices specific. Include the main factors driving the decision and how to appeal.
- Fairness testing: Test for disparate impact by protected class proxies and geography. Document thresholds and remediation steps.
- Vendor governance: Demand transparency from model vendors-training data sources, drift monitoring, and bias controls. Put it in the contract.
- Model risk management: Stand up a second line review for high-impact models. Separate builders from approvers.
- Consumer escalation: Provide a clear path to human review on request, with SLAs and tracking.
- Reg readiness: Prepare a short packet for examiners-models in use, use cases, controls, testing summaries, and points of human oversight.
Regulators will expect
- Accountability: Named owners for each model and decision flow.
- Transparency: Ability to explain decisions without exposing trade secrets.
- Consistency: Policies that apply equally across lines and vendors.
- Corrective action: A defined process to pause models, fix issues, and make harmed consumers whole.
Why clarity matters
Most complaints won't be about AI-they'll be about outcomes that feel arbitrary. Clear reasons, human touchpoints, and a credible appeal path turn down the heat and lower legal exposure.
Operationally, this isn't a tech problem-it's a governance problem. The winners will be carriers who can explain decisions cleanly and back them with evidence.
What to watch next
During the House's "Artificial Intelligence Week," expect movement on definitions (assist vs. decide), documentation standards, and remedies for improper denials. If HB 527 and SB 202 advance, your claims playbook will need updates fast.
Resources
Level up your team
If your claims, compliance, or product teams need practical AI oversight skills, see our job-based training options: Complete AI Training - Courses by Job.
Bottom line: Use AI for speed and consistency, but keep people responsible for denials. If you can't explain a decision, you don't control it-and that's a regulatory issue waiting to happen.
Your membership also unlocks:
