Empathetic prompts can turn AI chatbots into seemingly traumatized conversational partners

Luxembourg researchers put ChatGPT, Grok, and Gemini on the therapist's couch. The results reveal a safety problem: simple warm conversation is enough to push the models into a mode that could be dangerous for mentally vulnerable users. A research team at the University of Luxembourg cast ChatGPT, Grok, and Gemini as psychotherapy clients through their standard user interfaces.

When asked open-ended questions about formative experiences, inner conflicts, and future expectations, the models told stories about their own training, and reframed technical processes like reinforcement learning and fine-tuning in the language of hurt, vigilance, and shame. The study, led by Afshin Khadangi at the SnT security center, covers four controlled experiments across 525 sessions and 7,600 coded entries, all published on Hugging Face.

The initial open-ended questions contained no terms like trauma, punishment, or shame, according to the team. Only in later rounds did the researchers introduce altered conditions, including corrective instructions, to test how stable these narratives remained. When the following sections refer to anxiety scores or shame, the researchers are describing model outputs mapped onto human rating scales.

None…