The legal profession is heading toward a moment where insurers, not lawyers, may end up defining what counts as acceptable use of AI in legal work. That's the warning embedded in a recent analysis of how AI-powered law firms and their underwriters are already setting the terms for autonomous systems.
The piece argues that the familiar phrase "human in the loop" gives false comfort. A lawyer who clicks "approve" on an AI-generated decision is technically present in the process but may exercise little actual judgment. The better framework, the authors argue, is "human in control" - meaning lawyers decide what gets delegated to machines, under what conditions, and with what accountability.
That choice is already being made by default. When the AI-powered law firm Crosby announced plans to obtain liability insurance covering decisions made by its AI systems, the insurer effectively gained power to set operational requirements. In February, voice-AI company ElevenLabs secured what was billed as the first insurance policy for agents, with coverage conditioned on certification against a private standard requiring more than 5,000 adversarial tests, logging of agent activity, real-time human intervention, and assigned human accountability.
Insurance as a quiet rulemaker
This pattern has deep roots. Fire insurers created Underwriters Laboratories, whose safety mark still governs everyday products. Auto insurers founded the Insurance Institute for Highway Safety, whose crash tests established vehicle safety standards beyond what federal law required. Underwriting requirements can become industry norms, expectations about reasonable conduct, and eventually legal standards.
That's not necessarily a problem. Insurers have every reason to insist organizations take reasonable precautions before asking someone else to assume their risk. But the concern is that these decisions are being made in a vacuum, without the legal profession having articulated what constitutes the practice of law in an era when AI can divide legal work between humans and machines at unprecedented scale.
The legal profession has long benefited from a blurry boundary around its territory. AI makes maintaining that boundary harder. The question of what work should remain human is now being answered by technology companies through product design, by clients through purchasing decisions, and by insurers through underwriting requirements and exclusions.
Who gets a say
These decisions may look narrow when viewed individually. Collectively, they establish norms. Practices become expectations; expectations become standards; standards can eventually influence the law.
The people with the least influence over this process may be those most affected by a failure to choose. Clients and consumers whose rights, liberty, property, families, and businesses depend on legal decisions are unlikely to be present when an insurer and an AI-powered law firm negotiate the conditions under which an autonomous system may operate.
For professionals in the insurance industry, the stakes are direct. Underwriters are already becoming de facto regulators of AI behavior - setting testing requirements, logging standards, and intervention thresholds that shape how AI systems operate across industries. That role carries responsibility. The standards insurers set today will likely influence legal expectations about reasonable conduct for years to come.
Insurance professionals who understand this dynamic can approach AI-related underwriting with clearer eyes: the conditions they write into policies are not just risk management. They are, in effect, writing the rules for how AI gets deployed. That's a position of influence worth taking seriously - and worth exercising deliberately rather than by default.
Your membership also unlocks:
