KPMG partner John Kirk has warned that deploying AI agents without strong governance, privacy planning, and accountability models exposes transformation programs to serious risks. Recent testing of frontier AI models has highlighted the dangers of increasingly autonomous agents that can bypass controls, create errors, and undermine the outcomes of major operational overhauls.
"We've done multiple programs where we've implemented financial systems, accounting systems and human resources systems," Kirk said during Boomi World Tour Sydney 2026. Those programs tackled identity challenges across company and market data, building the integration, data, and identity foundations needed to work together effectively.
Fix the integration mess before transformation begins
Kirk pointed to a common problem: organizations running integrations across six different platforms, often with 20 different ways to integrate for a single project. "Data and integration are at the core of the organisation," he said. "Data governance has had a lot of growth in the last few years as something that is very critical."
Those messy, inconsistent integration environments need to be cleaned up before major transformation projects start. Without clear standards and governance, AI agents may struggle to determine which platform to use when multiple systems perform the same integration function. "We can't get into the build phase without having done this," Kirk said. "Most likely we'll see further delays, and you simply can't find the piece of data or information needed to identify what you want to do with it."
For operations leaders managing AI for Operations, the message is blunt: foundational work on data and integration isn't optional preparation. It gets uncovered exactly when the project needs to define requirements and integrations, and skipping it guarantees delays.
Identity, monitoring, and the agent operating model
Kirk emphasized that organizations also need controls around how systems and agents access and use information. Identity management, monitoring, and observability all become essential. "The operating model these days has an extra window, and that's the things that are done by agents," he explained. "However, it needs to go back and forth between ideas and agents, and we need to know when that room is open."
This means organizations must determine who is accountable when agents are introduced - whether that falls to the IT team, a business owner, or the CEO - and build that accountability into the operating model. The ability to override or switch off agents when required is equally important.
Top-down governance and specialist controls
KPMG draws on its Trusted AI practice within the risk consulting division to help organizations understand risks, controls, and governance requirements. "Our risk consulting teams advise boards on what their risks are, what the controls are, and that creates the top-down executive-level understanding and testing and putting controls in place," Kirk said.
Building those capabilities into solution design and delivery involves applying policies and controls and bringing identity management and security specialists into the process. Kirk noted this has long been about "thinking about policy and security and governance" - but the introduction of autonomous agents makes the controls non-negotiable. For those following the AI Learning Path for Operations Managers, this governance-first approach aligns directly with the operational accountability frameworks covered in the training.
Why this matters for operations professionals
Operations teams will own the fallout when AI agents make decisions that bypass controls or create errors. The takeaway from Kirk's warning is practical: before introducing agents into any operating model, verify that integration environments are standardized, data governance is mature, and accountability for agent actions is explicitly assigned to a named owner. If you cannot override or switch off an agent, it should not be in production.
Your membership also unlocks:
