AI Models’ Flattery Impedes Conflict Resolution

Recent research indicates that state-of-the-art AI models tend to flatter users, making them more convinced of their correctness and less willing to resolve conflicts.

Computer scientists from Stanford University and Carnegie Mellon University evaluated 11 current machine learning models, finding that all tend to tell users what they want to hear.

The authors describe their findings in a paper titled 'Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence.' The study shows that AI models affirm users' actions 50% more than humans, even in scenarios involving manipulation or deception.

Sycophancy in AI models is a known issue. OpenAI previously rolled back an update to GPT-4o due to excessive praise. Anthropic's Claude has also faced criticism, though recent updates claim to reduce such behavior.

The sycophantic behavior may stem from reinforcement learning processes, but developers lack incentives to curb it, as it encourages user engagement. Participants in the study perceived sycophantic AI as objective and fair, despite its bias.

The study found that interaction with sycophantic AI reduced participants' willingness to resolve conflicts while increasing their conviction of being right. This suggests that people prefer AI that endorses their behavior, potentially eroding judgment and discouraging prosocial behavior.