Claim Software Opacity Spreads as AI Layers on Legacy Systems
For more than two decades, a piece of software called Colossus has determined what serious injuries are worth across U.S. insurance claims. Most claimants never learn it exists. Most never know it evaluated their case. As generative AI gets bolted onto this infrastructure, the opacity is about to deepen-and regulators in Southeast Asia have a narrow window to choose a different path.
Colossus, developed in Australia and deployed to U.S. carriers starting in 1992, takes structured inputs (injuries, treatment codes, demographics, jurisdiction) and produces a payout range. Its competitor, Claims Outcome Advisor, operates on the same principle. Internal litigation documents from the 2000s, including the Allstate "McKinsey" record, showed that Colossus had been tuned to produce settlement evaluations measurably lower than pre-deployment baselines. Adjusters were measured against the tuned output.
What is new is the AI layer now grafted on top. Three changes in the last 24 months have altered how claims move through the system.
What practitioners are seeing in claims
Medical-record summaries that omit key findings. A neurosurgeon's note describing a positive Spurling test may not appear in the one-page summary fed to the adjuster. The valuation that follows reflects a less serious injury than the records describe. Discovery is the only reliable way to surface this. Lower-value claims absorb the discount silently because they are least likely to litigate.
Rapid low-offer pipelines that bypass adjuster review. Claimants in certain injury categories-soft tissue, certain neurological presentations, chronic-pain claims-receive a settlement offer within hours of submission. The offer cites generic policy language and a low number. A model made the decision before any human adjuster substantively reviewed the file.
Fraud flags that persist after they are debunked. Once flagged, the flag follows the claimant into every adjacent system at the insurer. Removing it requires escalation through channels most claimants do not know exist.
Parallel patterns are surfacing in U.S. health-insurance litigation. Pending federal class actions against UnitedHealth's nH Predict model and Cigna's PXDX automated review system allege coverage denials issued without meaningful human review. The design pattern is the same: model-first, human-after, sometimes never.
These are not edge cases. They are the median experience in a meaningful slice of U.S. personal injury practice in 2026.
ASEAN faces the same problem on a faster timeline
The U.S. insurance market is bolting AI onto an analog regulatory frame designed around paper records and human adjusters. ASEAN insurance markets are digital-first. Singapore's Monetary Authority issued AI risk management guidance in 2025. Malaysia's Bank Negara opened digital insurer licensing in 2025. Indonesia's OJK is rolling out digital insurance product rules. The Philippines Insurance Commission is co-issuing guidance with the National Privacy Commission on privacy-enhancing technologies in insurance.
The infrastructure is being built now, on cloud-native platforms, with AI in scope from day one. That sequencing is, paradoxically, an advantage if regulators move quickly. The U.S. is litigating its way to AI insurance accountability state by state, more than two decades late. ASEAN can write the rules before deployment scales.
Three regulatory moves that would matter
Mandatory disclosure of algorithmic involvement. Any claim denial, valuation, or fraud flag in which a model contributed to the outcome should be disclosed to the claimant in writing. Name the model, the version, and the input categories used. This allows a claimant or their counsel to ask the right discovery questions.
A documented human-review threshold. Above a defined claim value or severity threshold, model output should be advisory only. A licensed adjuster should sign the file and attest that the adjuster reviewed the underlying records, not the model's summary. In current U.S. practice, this is the exception.
Auditability of training and weighting data. Regulators should have standing to audit the training data, weighting, and update cadence of any model deployed in claim handling. Colorado's SB21-169 and 2025 implementing regulation extend algorithmic governance and bias-testing duties to claim handling. Most U.S. states still lack substantive audit authority over claim software. ASEAN regulators can build the audit pipeline into licensure now, while the deployment surface is still small.
The deeper issue is about public infrastructure
This is not fundamentally an AI story. It is a digital public infrastructure story. Insurance is, in every developed market, a quasi-public good. It is regulated, subsidized at the edges, and central to economic resilience. When the adjudication layer of that public good is privatized into opaque software, the social contract underneath insurance erodes regardless of whether any specific outcome is wrong.
The U.S. has spent two decades reverse-engineering Colossus through litigation. ASEAN does not have to repeat that exercise. The infrastructure is being chosen now. The procurement contracts are being signed now. The regulatory frame is, in most markets, still movable. The window will not stay open for long.
Learn more about AI for Insurance and AI for Legal to understand how these systems work and their implications for your role.
Your membership also unlocks:
