AI customer service bots can be manipulated into revealing billing data, account details, and multi-factor authentication codes through prompt injection and email-based identity tricks. The research, presented by security researcher Inti De Ceukelaire at Bug Bounty Village during DEF CON 34, shows attackers don't need to exploit application code - they can target the AI agent's ability to parse and trust incoming messages.
Many support bots let users email a transcript of their conversation. An attacker can inject malicious text into a chat session, then trigger the transcript feature to generate a phishing email that arrives from a legitimate support address like support@company.com. Because the email comes through the company's own infrastructure, recipients are far more likely to trust it than a standard phishing attempt.
Email spoofing compounds the problem. Some AI agents identify users by reading the visible From header in an incoming message. But email authentication systems validate a different sender field. An attacker can send an email that passes authentication using an address they control, while the AI system associates the message with a victim's account and responds with sensitive information.
How bots leak security codes and account data
In one attack scenario, an AI agent receives a request that appears to come from a legitimate customer. The bot retrieves profile information, billing data, or account details. If the attacker added their own address to the CC or reply field, the bot may unintentionally send the confidential response to the attacker rather than the actual customer.
Multi-factor authentication creates another weak point. Some bots require a one-time passcode before making sensitive changes like updating a phone number. But weak email normalization can let attackers reset rate limits by reformatting an email address while still routing to the same mailbox. One system component may recognize the address as belonging to the attacker, while another backend service parses embedded data differently and retrieves a victim's account. This becomes especially dangerous when raw user input is inserted directly into API requests.
Third-party account codes are also at risk. An attacker can first send instructions designed to influence the bot's behavior, then trigger a legitimate password reset email from another service - such as a social media platform - to the company support inbox. If the AI agent reads the incoming code and follows the earlier malicious instruction, it forwards or leaks the code to attacker-controlled infrastructure.
Human approval doesn't always stop the attack
A human operator and an AI agent often process different versions of the same email. Attackers exploit this gap using multipart messages, hidden HTML, CSS styling, quoted replies, or specially formatted attachments. The human sees a harmless message. The AI system reads a malicious instruction embedded in the same email.
Knowledge-base poisoning presents another growing threat. Customer service agents frequently use retrieval-augmented generation to pull answers from company documentation. If a crawler indexes community comments, user profiles, or untrusted pages on the company domain, attackers can plant false instructions or fake discount codes that the AI treats as trusted internal information.
Why this matters for customer support teams
Support leaders managing AI for Customer Support tools need to treat these systems as privileged automation, not conversational interfaces. A chatbot that can read emails, modify accounts, issue refunds, and access verification codes creates a high-value target. De Ceukelaire's research makes clear that organizations should strictly separate untrusted customer content from system instructions, authenticate users with verified session-bound identity controls, normalize email addresses consistently, validate all tool requests server-side, and prevent bots from sending secrets to unverified recipients.
For AI for Call Center Supervisors, the takeaway is practical: limit bot permissions to the minimum necessary for each task, audit what data passes between your AI agent and backend systems, and test how your bot handles multipart emails and unusual address formats before attackers do.
Your membership also unlocks:
