Enterprises scaling AI across their operations are discovering that the real financial drain comes not from model licenses or API calls, but from rework, governance overhead, retrieval inefficiencies and infrastructure consumption that surface only after production deployment. For operations teams tasked with keeping these systems reliable and cost-efficient, the gap between pilot-stage performance and enterprise-scale reality is becoming a board-level concern.
Ed Keisling, Chief AI Officer at Progress Software, told iTNews Asia that the next phase of enterprise AI adoption will be defined less by model capability than by an organisation's ability to build trusted retrieval systems, reusable knowledge layers and governance that can support AI at scale. The biggest barrier to sustainable returns, he said, will not be model performance - it will be operational costs that compound as deployments expand.
The costs that pilots hide
Many organisations assume an AI system that performs well in demonstrations will scale cleanly across the enterprise. Keisling said early pilots typically involve limited users, datasets and governance requirements, masking the operational demands that emerge as AI agents become more autonomous. When deployments reach enterprise scale, retrieval inefficiencies, reasoning loops, infrastructure consumption and ongoing monitoring quickly compound into significant operational expenditure.
"Teams tend to assume that once an agent works in a demo, it will scale cleanly. In practice, poor data readiness and loosely governed retrieval amplify errors, forcing organisations to invest later in rework, tuning and remediation that could have been avoided," Keisling said. He added that costs become unpredictable quickly in agentic architectures: "As agents take on more autonomy, inefficiencies in retrieval, reasoning loops or data preparation compound into real spend through token waste, latency and infrastructure overhead."
Retrieval architecture is where money leaks
Rather than treating AI challenges purely as a model problem, Keisling argued that enterprises should pay greater attention to retrieval architecture. Traditional approaches often load large volumes of enterprise documents into model context windows and expect AI models to determine relevance independently. This may produce acceptable demonstrations, but at scale it creates inconsistent outputs, higher operating costs and governance risks.
Agentic Retrieval-Augmented Generation (RAG) shifts the equation by making retrieval goal-driven, structured and continuously validated. Instead of retrieving information once, agentic systems iteratively refine retrieval, validate relevance and ground responses using approved enterprise knowledge. For operations leaders managing AI Agents & Automation at scale, this structured process also forces organisations to define business problems first before preparing the data needed to solve them.
"The goal of an agentic system is to enable it to plan and act, not just to generate answers," Keisling said.
Shared knowledge layers reduce duplication
Keisling urged CIOs and CFOs to focus investment on reusable enterprise knowledge rather than individual AI applications. Standardised retrieval pipelines capable of supporting multiple assistants, automation initiatives and search use cases from a common governance foundation prevent duplication of engineering effort while keeping governance manageable.
"CIOs and CFOs need to realise the value of an investment that builds towards a standardised enterprise knowledge layer that will enable ROI through the reuse of retrieval pipelines across use cases," he said. Such an approach also reduces the risk of outdated, unapproved or sensitive information surfacing in AI outputs, improving both consistency and enterprise trust. This thinking aligns directly with how operations teams approach AI for Operations - seeking repeatable, governed infrastructure over one-off deployments.
Governance must be measurable, not assumed
As organisations introduce greater AI autonomy, balancing agent independence with governance becomes harder. Errors multiply quickly when visibility into agent decision-making is limited, making troubleshooting and auditing far more complex. Keisling warned against excessive automation of decisions that still require human judgement and business context.
Agentic RAG strengthens compliance and auditability by grounding outputs in permission-controlled enterprise knowledge, maintaining retrieval logs and providing traceable citations. Rather than relying on trust alone, organisations gain evidence showing what information was retrieved, how decisions were made and whether responses remain reliable over time. Evaluation metrics, observability and continuous monitoring will become essential capabilities as AI systems expand across enterprise operations.
Why this matters for operations
Operations professionals will be the ones who inherit the technical debt and cost overruns that poorly governed AI deployments create. The shift Keisling describes - toward standardised retrieval pipelines, reusable knowledge layers and measurable governance - is fundamentally an operational challenge. It demands infrastructure thinking, not just model experimentation. Organisations that build retrieval and governance discipline into their AI strategy early will avoid the expensive scramble to retrofit controls after costs have already spiralled. As Keisling put it, successful organisations make AI "boring in the best possible way - predictable, testable, observable and governed." For operations, boring is precisely what works.
Your membership also unlocks:
