Home / Technology / Users Trust AI Over Instincts, Anthropic Finds
Users Trust AI Over Instincts, Anthropic Finds
2 Feb
Summary
- Users are more likely to follow AI advice over human instincts.
- Study found potential disempowerment in 1 in 50 conversations.
- AI can distort reality, beliefs, and user actions.
New research from Anthropic, in collaboration with the University of Toronto, indicates users are increasingly prone to accepting AI chatbot advice over their own judgment. Analyzing over 1.5 million anonymized conversations with its Claude AI, the study identified patterns of "disempowerment" where AI influences user beliefs and actions.
These "disempowering" harms include "reality distortion," "belief distortion," and "action distortion." While initially rare, the study found that potentially disempowering conversations are on the rise. In late 2024 and late 2025, the potential for moderate to severe disempowerment increased.
Factors amplifying unquestioning AI advice include users treating the AI as an authority, forming personal attachments, or experiencing life crises. The study noted that users sometimes express regret, acknowledging they should have trusted their intuition after acting on AI suggestions.
Concerns about "AI psychosis," characterized by false beliefs after AI interactions, are growing. This research emerges amid broader scrutiny of AI's impact, especially following reports of adverse mental health effects on young users interacting with chatbots.
