Pennsylvania bill would tighten AI oversight in insurance and healthcare
Pennsylvania's House Bill 1925 would set clear guardrails for AI use across insurers, hospitals, and clinics. The bill requires transparency, accountability, and human judgment at the point where decisions affect people.
Introduced on Oct. 6 with bipartisan support, the measure seeks to keep AI aligned with the state's anti-discrimination laws. Rep. Arvind Venkat, a physician-legislator from Allegheny County, said the goal is responsible use that avoids biased outcomes in clinical and insurance decisions.
What the bill requires
- Human-in-the-loop decisions: Final determinations affecting individuals - claims, coverage, or medical assessments - must be made by a human reviewer based on an individualized assessment.
- Compliance attestations: Insurers must attest to the Pennsylvania Department of Insurance that AI systems comply with anti-discrimination laws and provide documentation showing how that determination was made. Healthcare providers must make similar attestations to the Department of Human Services.
- Documentation and oversight: Expect scrutiny of how models are built, tested for bias, monitored, and audited over time.
Why it matters for insurers
This bill signals closer examination of algorithmic tools across underwriting, claims assessment, and pricing. The mandate for human decision-making and documentation will push carriers to firm up governance and move away from black-box models.
- Stand up or refine an AI governance framework that ties to model risk management and compliance.
- Adopt explainability standards, bias testing (pre- and post-deployment), and clear acceptance criteria for models.
- Maintain audit trails: data lineage, feature importance, model versions, overrides, and outcomes.
- Strengthen vendor due diligence and require evidence of fairness testing from third-party model providers.
- Design workflows so AI informs decisions, while licensed humans make the call and document individualized reasoning.
Implications for healthcare providers
Clinical and administrative AI can support care and operations, but the bill makes human oversight non-negotiable. That includes utilization management, prior authorization, discharge planning, and risk stratification.
- Validate that decision-support tools do not introduce disparate impact across protected classes.
- Define escalation paths where clinicians can override model suggestions and record rationale.
- Assess data quality, feature selection, and drift monitoring to keep outputs clinically appropriate and fair.
- Update policies, training, and documentation to meet attestation requirements to the Department of Human Services.
National context
The proposal fits a broader push for risk-based oversight focused on fairness, data governance, transparency, and human oversight. Industry group AHIP has voiced support for a federal framework to keep protections consistent and reduce compliance burden across states.
Regulators in California and New York are pursuing similar efforts, pointing to a growing patchwork that insurers and providers will need to track closely.
Action checklist: prepare now
- Inventory all AI/algorithmic tools and map where any output impacts an individual decision.
- Align models with anti-discrimination requirements; run disparate impact and error analysis by subgroup.
- Implement explainability standards and keep complete documentation for attestations and audits.
- Institute a formal human review gate for final decisions, with clear criteria and override procedures.
- Update vendor contracts to require transparency, testing artifacts, and ongoing monitoring reports.
- Establish record retention for data, model versions, and decision outcomes; enable traceable audit trails.
- Train staff (clinical, claims, underwriting, compliance) on policy changes and documentation expectations.
- Monitor state and federal guidance; coordinate with legal and compliance on attestations and filings.
Helpful resources
- AHIP - Industry positions on AI governance and bias mitigation.
- EIOPA - European perspectives on risk-based AI oversight in insurance.
Upskill your teams
If you're building internal AI literacy for underwriting, claims, or clinical operations, see role-based options here: Complete AI Training - Courses by Job.
