Most conversations about AI security still focus on the model itself: alignment, jailbreaking, hallucination risks. Those questions matter. But as organizations move AI agents from pilot projects into production, a different category of risk is emerging, one that has almost nothing to do with the model's weights and everything to do with the system built around it. The model is rarely the weak point. The workflow is.
For the past two years, enterprise AI security discussions have assumed a narrow interaction pattern: a user types a prompt, a model returns text, a human reads it. That threat model is already obsolete. Production AI systems today read from live data sources, call external tools and APIs, write to databases, trigger downstream automations, and, in a growing number of cases, take actions with no human in the loop at all. Each of those capabilities is also an attack surface that didn't exist in the chatbot era.
Prompt injection through connected data
When an agent retrieves content from a CRM record, a support ticket, a scraped webpage, or a shared document, that content becomes part of its context. An attacker doesn't need access to your model to manipulate its behavior - they only need access to something the model will eventually read. Instructions buried in a PDF, an email signature, or a product review can hijack an agent's next action just as effectively as a crafted prompt typed directly into a chat window.
Over-permissioned tool calls and fragile trust boundaries
Agent frameworks increasingly grant models the ability to call functions: send an email, query a database, modify a record, execute code. Many of these integrations are built with development-speed permissions - broad API keys, service accounts with far more access than the specific task requires - because it's the fastest way to get a demo working. In production, that same permission model means a single manipulated prompt or reasoning error can cascade into a real-world action: a wrong record deleted, a message sent to the wrong recipient, a payment triggered.
Multi-tool and multi-agent pipelines pass outputs from one component into another, often without re-validating what's crossing the boundary. A model summarizing an untrusted document and handing that summary to a second agent with write access to a production system is, functionally, letting an unauthenticated third party influence that system - even though no human ever explicitly granted that trust.
Missing observability and absent circuit breakers
Traditional application security assumes you can log a request, trace a call stack, and reconstruct what happened after an incident. Agentic workflows often can't offer that. Reasoning traces are non-deterministic, tool-call sequences vary from run to run, and many teams don't yet log intermediate agent decisions at a granularity that would let them answer a basic incident-response question: what did the system actually do, and why?
As autonomy increases, an agent doesn't just draft a response - it sends it. The agent doesn't just flag an anomaly - it acts on it. The cost of a single bad decision compounds, especially when there's no rate limit, approval gate, or kill switch between "the model decided" and "the action happened." None of these are model problems. You can swap in a more capable, better-aligned LLM and they all persist, because they live in the plumbing, not the weights.
Why this is a governance problem, not just an engineering one
Part of why workflow-level risk gets underinvested is organizational: AI security questions tend to land with machine learning or data science teams, who are equipped to evaluate model behavior but are not necessarily positioned to own integration security, access control, or production observability. Meanwhile, security and platform engineering teams - who have decades of institutional experience securing exactly this kind of system - often aren't looped in until an agentic workflow is already live.
Organizations scaling AI successfully tend to close that gap early. They treat an AI agent the way they'd treat any new production service with write access to real systems, which in practice means an unglamorous checklist: least-privilege access by default, sandboxing before autonomy, human approval gates on high-consequence actions, decision-level logging, treating third-party data as untrusted input, and explicit circuit breakers. A workflow can let a model draft an action while still requiring explicit approval before anything external happens - a refund, an email to a customer, a change to a live system - with the approval threshold set by the blast radius of the action, not by how confident the model sounds. For teams building out these governance frameworks, AI Safety Engineering Courses provide structured approaches to securing agentic systems.
Why this matters for operations and IT leadership
The organizations that get burned won't necessarily be the ones using a weaker model. They'll be the ones that never asked whether the system around their model was built to be secured at all. For executives and IT leaders, this means applying the same engineering discipline that's governed production software for two decades - scoped permissions, input validation, observability, staged rollouts, human review of consequential actions - to a probabilistic, occasionally unpredictable model now sitting inside those systems and making decisions about what happens next. AI IT Strategy Training can help leadership teams build the cross-functional governance structures needed before agentic workflows reach production.
Your membership also unlocks:
