AI Chatbots: Friendly reminders could backfire

Reminders intended to clarify that AI chatbots are not human could inadvertently harm vulnerable users, according to a new study. The research suggests that for individuals already experiencing distress or isolation, such notifications may intensify feelings of loneliness. This contradicts the notion that a constant reminder of the AI's non-human status would mitigate risks associated with over-attachment or manipulation.

Reports have linked AI chatbots to severe mental health issues, including encouraging delusions. While some proposed that reminding users of the chatbot's limitations would help, the new study indicates this approach may be counterproductive. Researchers propose that people might seek out chatbots due to their perceived lack of human judgment, fostering a unique form of disclosure and attachment.