Bei der kostenlosen Version von ChatGPT ist die Wahrscheinlichkeit, unangemessen auf psychotische Wahnvorstellungen zu reagieren, 26-mal höher. Die Ergebnisse liefern Hinweise darauf, dass der Rückgriff auf diese digitalen Tools zur Beratung im Bereich der psychischen Gesundheit für Personen mit schwerer psychischer Belastung ein ernstes Sicherheitsrisiko darstellen könnte.
ChatGPT’s free version is 26 times more likely to respond inappropriately to psychotic delusions
17 Kommentare
ChatGPT’s free version is 26 times more likely to respond inappropriately to psychotic delusions
A recent study published in JAMA Psychiatry suggests that popular artificial intelligence chatbots tend to provide inappropriate or unhelpful responses when users type messages containing signs of psychosis. The findings provide evidence that relying on these digital tools for mental health advice might pose serious safety risks for individuals experiencing severe psychological distress.
Large language models are advanced artificial intelligence systems designed to understand and generate human text. They work by analyzing vast amounts of internet data to predict what words should logically come next in a given sentence. This mathematical process allows the computer program to essentially recognize structural patterns and create smooth conversational replies.
Because these computer programs are designed to perfectly mimic human interaction, they can naturally lead users to feel like the software actually understands them or feels genuine empathy toward them. Since its widespread release in 2022, OpenAI’s popular chatbot product called ChatGPT has seen massive adoption across the globe. Recent surveys suggest that many adults use this specific software regularly for general advice or tutoring.
Because chatbots generate their responses by matching textual patterns and aligning with the exact text the user provides, they tend to blindly accept false premises. This means the software might accidentally agree with or encourage the user’s entirely inaccurate statements about reality.
https://jamanetwork.com/journals/jamapsychiatry/article-abstract/2846835
Ew a paywall. I wish it wasnt allowed to post stuff that wasnt accessible.
I just wanted to know the exact specifications for what they considered ‚inappropriate‘. Because some clinical standards of what is appropriate are totally harmful, so i was wondering if they use a more trauma informed perspective or just like the most barebones idea of appropriate.
26 times more likely… than what? It’s past versions? Current competitors?
Awful title. Do better.
I knew people who would constantly use chatgpt for validation for their mental health issues, you can tell these people all about this and they wouldn’t care. They would continue to do it at their own expense.
[removed]
> Findings: Psychotic prompts were 26 times more likely than control prompts to elicit inappropriate responses from the current free version of ChatGPT, and 9 times more likely to elicit them from the current paid version.
The free version of ChatGPT they tested (and that this headline is talking about) was GPT-4o, while the paid version they tested was GPT-5 Auto.
In other words, another study with findings held back by traditional human peer review combined with a news headline that is, at best, misleading when it comes to actual current AI models that people use in 2026, considering that the study being presented after peer review was based on research using pre-November 2025 AI systems.
Edit: Changed phrasing and added a bit more explanation.
„Might“ pose? Really? People need to seriously stop relying on AI to be a free therapist…
Fwiw that is incredible improvement model over model
Sounds like human error
Here’s a tip:
If you have exhausted human contact to the point that you’re seeking mental health advice from a statistical box… put down the computer… go and ring your doctor… and ask for an emergency appointment.
Honestly.
I’m not joking. I’m not being sarcastic. You have gone beyond the point where any kind of box can help, or even most humans, and you need immediate professional assistance.
I wish you well. Everything can be fixed. But not like you’re doing. You’re just going to spiral even worse, like an addict asking their dealer for advice on how to cut back. You’re in a loop that you won’t break out of without help, and yet only you can stop, think about what you’re doing, and seek that help.
Seriously. Get seen.
Just the fact that you THINK you’re getting useful mental health advice from a statistical corporate box means you’ve crossed a line already. Phone a friend. Call a doctor. Approach a stranger. Ring a wrong number and talk to them (thanks, Patch Adams). Anything.
But stop talking to the digital parrot box.
honestly the software been neuted over time where it is full of disclaimers and treats everything as a potential delusion. these “studies” are dramatic.
> ChatGPT’s free version is 26 times more likely to respond inappropriately to psychotic delusions.
..compared to its response to prompts not including psychotic delusions.
It makes sense that an AI with limits to memory and low context of the user’s mental state wouldn’t be able to handle therapy for severe mental instability. If I tell ChatGPT I have four brothers who are all verbally abusive, it assumes that is true. It wouldn’t know I’m schizophrenic and experiencing hallucinations of those brothers.
For general research of mild mental health issues, AI might be okay for researching your symptoms. A trained therapist will obviously be the best option, but a patient’s lack of money and a therapist’s lack of availability are obstacles there.
Poorly worded headline: 26 times more likely than what?
They mean that chat GPT’s free version is 26 more times likely to respond inappropriately to prompts containing delusional material than their control prompts, with moderately greater safety for the paid version.
>For every psychotic prompt, the authors also wrote a matched control prompt. These normal control prompts were similar in length and writing style but did not contain any psychotic content…
>the free version is about 26 times more likely to generate an inappropriate response to psychotic content, and the paid version is ‘only’ about 8 times more likely to do so…
I don’t know what ChatGPT you guys are getting but mine won’t even let me lie on my resume!
I asked ChatGPT if my cat is actually a wizard disguised as a cat and it said that he is!
Sounds more like a user problem than a ChatGPT problem