Most contact centers deploying AI are skipping the foundational work that makes automation sustainable, according to Dave Rennyson. The rush to launch AI agents often ignores the new types of data these systems generate and the scientific rigor needed to measure their performance.
The hidden data stream from AI agents
AI agents produce more data than human agents, including failure signals and turn-taking data that human conversations never generate. Rennyson said this data represents a significant opportunity, but only if organizations build the architecture to capture and act on it. Without that infrastructure, contact centers miss early warning signs of performance degradation.
Why ground truth is the step most teams skip
Establishing a reliable baseline for AI performance-what Rennyson calls "ground truth"-demands real scientific rigor. "Ground truth is the step almost everyone skips," he said. When teams neglect this, model drift sets in. Small inaccuracies compound, and the AI starts handling conversations in ways that were never validated against a known standard.
Agentic AI rewrites the design rulebook
Agentic AI is genuinely different from legacy IVR systems. The removal of linear flow constraints opens up a new design space, but only if organizations build the right orchestration and monitoring layers on top. Without those layers, the flexibility becomes a liability, not an advantage.
Turning surveys into a real-time performance tool
Rather than declaring surveys dead, Rennyson said that generative AI can now appraise every single conversation at scale. This turns a historically biased metric-often based on a tiny fraction of interactions-into a far more effective measurement tool. By analyzing all conversations, teams can spot patterns that traditional surveys never captured.
Why this matters for customer support professionals
Support leaders should treat AI deployment as an ongoing data discipline, not a one-time software rollout. The metrics that matter are no longer just handle time or satisfaction scores; they now include model drift, failure rates, and the quality of the data the AI itself generates. Investing in the architecture to capture and analyze these signals will separate teams that sustain AI performance from those that see it degrade within months.
Your membership also unlocks:
