Business leaders are pushing agentic AI into production with "no idea how to manage risk," according to BCG's chief AI ethics officer Steven Mills, who warned yesterday that companies prioritizing speed over governance could see early AI gains unravel after a single incident.
Mills, a partner and managing director at Boston Consulting Group, said organizations face mounting pressure to demonstrate AI-driven productivity, yet their risk management programs remain "cumbersome and slow" - unable to keep pace with exponential AI scaling. The warning lands amid growing public alarm over AI safety, including a high-profile resignation from an Anthropic researcher the day prior.
The gap between deployment speed and risk controls
"The desire to move fast on AI without putting appropriate risk management in place can result in a system lapse or use cases being deployed in areas with strong regulatory requirements that organizations are unprepared for," Mills wrote in a blog post. He noted that many business leaders have told him directly that existing governance processes cannot match the speed their AI initiatives demand.
The consequences of getting governance wrong are not incremental, Mills argued. "Every bit of value you've built with experimentation and early wins could unravel because of a single incident," he wrote. That risk becomes more acute with agentic AI - systems that can act autonomously rather than simply generating text or analysis.
Broader safety concerns intensify
Mills' blog post appeared one day after Anthropic researcher Jacob Coxon resigned publicly, alleging that AI companies are "gambling with our lives" and warning that self-improving AI could kill all humans within a decade. Coxon joins a growing list of AI professionals who have left their roles over safety concerns. Mills did not reference Coxon or other resignations in his post.
The timing underscores a widening split between corporate AI adoption and the caution urged by safety researchers. While Mills focuses on enterprise governance failures, the broader alarm from figures like Coxon points to existential risks that extend far beyond regulatory compliance.
Why this matters for managers
If your organization is deploying agentic AI or planning to, the governance question is no longer theoretical. Mills is describing a real tension managers face daily: executives want speed, but risk frameworks haven't been adapted for autonomous systems. A single regulatory misstep or uncontrolled AI action can wipe out the efficiency gains you've been tasked with delivering. The practical takeaway is not to slow down - it's to push risk management to match the pace of deployment, before an incident forces the issue.
Your membership also unlocks:
