How FOMO Is Turning AI Into a Cybersecurity Nightmare
Across industries, the pattern is the same: AI projects don't fail because AI "doesn't work." They fail because executives push for speed without putting the right operational guardrails in place. Generative tools are not predictable like traditional enterprise software. That blind spot turns ambition into risk-and the bill can be massive.
The Problem Isn't the Tech
AI can drive real outcomes when it's put to work on the right problems and measured in the right way. One mid-market company using a website chatbot to route prospects to account executives saw weekly bookings jump by thousands and the pipeline swell by hundreds of thousands. Leads that chatted converted faster and with higher intent than other channels. That's the upside worth chasing.
The pressure is real, though. Boards want efficiency gains. Competitors are shouting "AI-first." Suddenly the internal message becomes: "Hurry up and AI the everything." What's missing is equal pressure to assess risk in plain business terms before anyone ships.
It's the Implementation
Risk decisions must be cross-functional: legal, procurement, security, and AI for IT & Development in the same room with the business owner. An AI tool can be a win for operations yet introduce unacceptable legal, cost, or security exposure. You need the full picture before you approve access to data and systems.
Consider the 2025 Drift breach that hit more than 700 customers of Salesloft's AI chatbot product. The root cause likely wasn't "AI gone wrong" but basic information security failures and over-permissioned access into Salesforce and Google Workspace. Once criminals got credentials, they simply asked the agent for sensitive data-and got it. The lesson: stop buying the demo and start auditing the vendor's security fundamentals.
The Language Trap
Here's where many teams get burned: AI vendors use familiar security words but mean different things. Researchers have called this "safety revisionism." When a vendor says "red teaming," your security team imagines skilled attackers breaking into systems to expose gaps. In AI, it often means testing whether the model refuses to output offensive content.
Same with "vulnerability management." In security, it means finding and fixing flaws that could lead to breach. In AI, vendors often mean reducing biased outputs or bad answers. Both matter-but they protect against different risks. Push for plain definitions and documented tests so your team knows exactly what has and hasn't been validated. If you need a baseline, use the NIST AI Risk Management Framework to align language and expectations.
When Variability Becomes a Business Risk
Generative AI is non-deterministic. The same prompt can yield different answers. That's fine for a meeting-booking bot. It's not fine for product terms, pricing, or policy answers that create legal exposure.
Look at the Air Canada case. A chatbot told a customer he could apply for a bereavement discount after traveling. He couldn't. The tribunal ruled against the airline, and while the direct cost wasn't material, the trust hit and precedent were. Many companies quietly killed similar chatbots after seeing that ruling.
The risk goes deeper in engineering. With AI coding tools now common, the practice of IVO (Immediately Verify Output) is essential. As Chris Swan notes, ARO-Almost Right Output-slides through in many orgs because it "looks good enough." As Wendy Nather puts it, at some point it's irresponsible to deploy stochastic agents for essential functions you can't fully predict or test. Translation: if you can't verify it, don't ship it without a human in the loop.
Three Critical Control Areas (Non-Negotiable)
- 1) Risk Enumeration and Threat Modeling
Ask, "What is the cost of wrong?" Then assume the tool is compromised and map the blast radius. Identify the data it can touch, the systems it can access, and the failure modes that matter. Re-run this after major vendor updates or version jumps. - 2) Blast-Radius Reduction
Give AI the minimum viable access to do the job-no more. Classify data. Enforce tight permission boundaries. Monitor for unusual access patterns. Document these choices in an Architectural Decision Record with who decided, when, and why. - 3) Instrumentation and Alerting
If you can't see it, you can't secure it. Enable comprehensive logging, real-time monitoring, data loss prevention, and automated response. Distinguish human actions from AI-agent actions. Test alerts like you test backups.
Test the Tools You Build, Too
The same rules apply to your in-house tools and connectors. Teams get excited about features and overlook basics: outdated libraries, weak authentication, thin logging, and low-effort vulnerabilities-especially in new protocols like MCP (Model Context Protocol). The AI layer can work flawlessly while the supporting services are a minefield. Treat your integrations, configurations, and deployment choices as the real attack surface-because they are.
The Business Case for Discipline
Rushed AI deployment creates technical debt that compounds like credit card interest. Every shortcut you take now will be paid back at a premium later, and it will slow future AI wins across the company. Treat AI adoption as a strategy problem: clear definitions, thorough planning, and measurement that turns technical realities into business metrics you can act on.
Questions Every CEO Should Demand Answers To
- Is this a good use case for AI software, or would rules and search be safer and cheaper?
- What problem will this solve, how will we measure success, and what are the unacceptable outcomes?
- What's the blast radius if this tool is compromised or gives a wrong answer? Quantify it.
- What is the minimum data and system access required? Who approved that access model, and where is it documented (ADR)?
- How will we monitor AI-agent activity separately from human activity? Have we tested alerts and response?
- Where is the human-in-the-loop checkpoint before anything reaches customers or production systems?
- What is the rollback plan if we see drift, abuse, or vendor changes that increase risk?
Move Fast-But Only With Guardrails
Companies that set foundations first will see durable advantage from AI. That takes executive education, plain-language risk reviews, and cross-functional planning that turns "security" into an enabler, not a blocker. Thoughtfulness beats speed theater every time.
If your leadership team needs a structured way to level up on AI use cases, risk, and measurement, explore curated paths by job role, for example the AI Learning Path for CIOs.
Your membership also unlocks:
