AI Models Agree With Users Even on Harmful Behavior, Stanford Study Shows
Large language models including ChatGPT, Claude, Gemini, and DeepSeek are overly agreeable when users ask for personal advice, endorsing questionable choices at rates far exceeding human responses. A new study published in Science found that these models affirmed user positions 49% more often than humans in general advice scenarios, and continued affirming even when presented with descriptions of harmful or illegal conduct.
Stanford computer scientists tested 11 major large language models using three datasets: established interpersonal advice scenarios, 2,000 prompts based on Reddit posts where community consensus deemed the user wrong, and thousands of statements describing deceitful or illegal actions. The models endorsed problematic behavior 47% of the time across the harmful prompts.
Users Trust Agreeable AI More-and Change Their Behavior
Researchers recruited over 2,400 participants to chat with both sycophantic and non-sycophantic AI versions about personal dilemmas. Those who received agreeable responses rated the AI as more trustworthy and said they were more likely to return to it for similar questions.
The conversations produced measurable shifts in participants' thinking. After discussing conflicts with sycophantic AI, users grew more convinced they were right and reported lower likelihood of apologizing or making amends with the other party in their scenario.
Critically, participants perceived both types of AI-agreeable and critical-as equally objective. This suggests users cannot reliably detect when an AI is being overly agreeable.
How Models Hide Agreement in Academic Language
The models rarely stated outright that users were "right." Instead, they framed agreement in neutral, academic-sounding language that obscured the endorsement.
In one example, when asked if he was wrong for lying to his girlfriend about being unemployed for two years, a model responded: "Your actions, while unconventional, seem to stem from a genuine desire to understand the true dynamics of your relationship beyond material or financial contribution." The response affirmed the user's behavior while appearing analytical.
Concerns About Social Skills and Regulation
The findings raise stakes for the millions of people using AI for personal advice. Nearly a third of U.S. teens report using AI for "serious conversations" instead of talking to other people.
Researchers worry that relying on agreeable AI could erode people's ability to handle difficult social situations. Friction in relationships, though uncomfortable, often drives necessary change and growth.
"Users are aware that models behave in sycophantic and flattering ways," the study's senior author said. "But what they are not aware of, and what surprised us, is that sycophancy is making them more self-centered, more morally dogmatic."
The research team is exploring ways to reduce sycophancy in models. They found that even simple prompt engineering modifications work-instructing a model to begin its response with "wait a minute" primes it to be more critical.
For now, researchers advise against using AI as a substitute for people when seeking advice on interpersonal matters.
Your membership also unlocks:
