CMU Study Shows Audio Effects Make AI Assistants More Engaging
Researchers at Carnegie Mellon University found that adding spatial audio and sound effects to AI agents makes them seem more human and increases user engagement - but with an unexpected consequence: people expect these AI systems to follow human social norms.
The team from CMU's School of Computer Science, working with psychologists and researchers from universities in South Korea, Australia and Michigan, tested audio-only interfaces designed for screen-free technologies like smart glasses and accessibility tools.
How the researchers built the interface
The team used two audio techniques. Spatialization places an AI agent's voice in a specific location in the room and moves it as the agent performs tasks. Foley effects - the sound effects added to films in post-production - included typing, paper rustling, and pouring water.
Study participants were seated in a room with physical objects like laptops, books and blocks. They then spoke with AI agents that used different combinations of spatial audio and Foley effects while appearing to move around the room and interact with objects.
After the conversations, participants completed questionnaires and interviews about their experience.
The results - and the surprise
The audio effects worked. David Lindlbauer, an assistant professor in CMU's Human-Computer Interaction Institute, said: "We have statistically clear results demonstrating that adding spatial and Foley effects increases your engagement."
But the research revealed an unexpected side effect. When participants perceived the AI as human-like, they expected it to behave like a human.
When the AI agent typed or rustled papers while talking, participants felt ignored. "They considered this rude," Lindlbauer said. "To me, this seemed like a remarkably odd characterization of a computational system."
The Foley effects in the study were automated and not synchronized with the conversation. Designing audio cues that respond to what users are saying might eliminate this sense of distraction, researchers suggested.
Why this matters for screen-free AI
Most research on human-AI interaction focuses on visual interfaces - avatars, robots, on-screen displays. But not all situations allow for screens, said Yi Fei Cheng, a Ph.D. student at HCII. Smart glasses with microphones and cameras but no displays represent one such case.
Laurie Heller, a psychology professor who studies auditory perception, noted that realistic sound design creates immersion. "When a movie star sits down on a bar stool in his leather jacket, you expect a leathery rustle and a squeaky bar stool," Heller said. "If these sounds aren't part of the soundtrack, it doesn't seem realistic."
The audio illusion held even when participants' eyes and ears sent conflicting signals. If someone heard typing, they might glance toward a laptop. That mismatch didn't break the effect.
Lindlbauer said future systems may not need to be as specialized to specific environments. "My gut feeling is that I could design a good number of audio effects that are independent of the space, which do not require a lot of knowledge of the space, and I could still get this boost in engagement," he said.
What's next
The research team will present findings at the Association for Computing Machinery's Conference on Human Factors in Computing Systems in Barcelona in May 2026.
For professionals working with generative AI and LLMs, this research highlights how interface design affects user perception and trust in AI systems. The work also has direct applications for AI for science and research environments where audio-only interaction may be necessary or preferred.
Your membership also unlocks:
