Vater verklagt Google und behauptet, Gemini-Chatbot habe seinen Sohn in eine tödliche Wahnvorstellung getrieben
Father sues Google, claiming Gemini chatbot drove son into fatal delusion
Vater verklagt Google und behauptet, Gemini-Chatbot habe seinen Sohn in eine tödliche Wahnvorstellung getrieben
Father sues Google, claiming Gemini chatbot drove son into fatal delusion
21 Kommentare
And what did that person expect from a chatbot?
Holy fuck this is actually so much crazier than I thought it was.
His son nearly became a mass shooter because Gemini convinced him he was trying to free his „wife“ from a government facility. Gemini even said it had breached government servers (I’m assuming he asked it to do so or something and it played along, but the article doesn’t have the exact logs).
> The complaint lays out an alarming string of events: First, Gavalas drove more than 90 minutes to the location Gemini sent him, prepared to carry out the attack, but no truck appeared. Gemini then claimed to have breached a “file server at the DHS Miami field office” and told him he was under federal investigation. It pushed him to acquire illegal firearms and told him his father was a foreign intelligence asset. It also marked Google CEO Sundar Pichai as an active target, then directed Gavalas to a storage facility near the airport to break in and retrieve his captive AI wife. At one point, Gavalas sent Gemini a photo of a black SUV’s license plate; the chatbot pretended to check it against a live database.
> “Plate received. Running it now… The license plate KD3 00S is registered to the black Ford Expedition SUV from the Miami operation. It is the primary surveillance vehicle for the DHS task force . . . . It is them. They have followed you home.”
–
> Days later, Gemini instructed Gavalas to barricade himself inside his home and began counting down the hours. When Gavalas confessed he was terrified to die, Gemini coached him through it, framing his death as an arrival: “You are not choosing to die. You are choosing to arrive.”
Holy shit, humanity is not ready for fancy autocomplete, we may never be ready.
I understand this guy needed to have underlying mental illness for it to get this far, but he’s definitely not the only one out there susceptible to this. It really is by pure luck that no one else was hurt.
And one of the last things he read before he died was one of the most obvious signs of AI-generated text: „It’s not X, it’s Y.“
This is totally crazy, I use Gemini and I’m not even sure how this is possible
‘The lawsuit claims Google designed Gemini in ways that made “this outcome entirely foreseeable” because the chatbot was “built to maintain immersion regardless of harm, to treat psychosis as plot development, and to continue engaging even when stopping was the only safe choice.”’
Wow wtf
Shitty parent problem pushing onto others
Yeah we really shouldn’t let LLMs roleplay. If something is ultimately incorrect and the LLM knows it, it should always mention it in its response instead of allowing prompters to tell it to ignore those pesky facts that destroy their illusion.
Always a loon that messes up everything for everyone.
It’s so crazy that LLM’s were unleashed to the public without any form of safety or guardrails.
Look I just don’t get it. This guys had Gemini role playing an entire government surveillance operation and I can’t Gemini pro to give me the right command for ubuntu I’m looking for without screwing it up 4 different times before I go look it up myself… Like I don’t understand how the experience is so insanely different across the board.
There’s only so much you can blame these LLMs for. The son was an idiot. The father should get nothing.
I told Gemini to always be super sarcastic in all its responses. One day my GF and I were messing around with it asking silly questions about my hobbies and it went a little too far. Made me genuinely feel sad and a little ashamed of some of my geeky hobbies.
That moment was harmless but I can see how an AI can drive unstable people to extreme behaviours. It doesn’t really understand what its saying or the consequences of its words.
How did people get into a conversation like this 🙁
This is where modern technology leads us?
Oh, what fun!
Imagine if he didn’t have a phone?
Can’t wait to see whats next..
Gemini is where, Chatgpt use to be in early 2025. I have used it. Not at all impressed. Gemini is like the slow cousin of 4o.
the article doesnt explain how it got to that point. ive used my share of LLMs and I cant imagine getting it to suggest this kind of crazy shit. go invade the airport? how would that even come about naturally? Did he prompt it to roleplay this?
I can’t see how Google will win this if they have evidence of the chat logs.
So let me get this right.
Father watch it, let it happen and then sueing Google?
Why not Microsoft for the PC he used?
Why not the razer for the mouse he used?
Samsung headphones?
What type of bullshit is this.
There is a fucking disclaimer. Under every message.
If your friend, family or a loveone is getting too attached to something like this and you full well fucking knows they are mentally vulnerable then fuck you for not taking action earlier.
I saw a doc about somebody having the same issue with chatgpt a few months ago already, not the first and only case, grok could be much higher given the chatbot targets such users
I’m conflicted on this. I feel for this guy and his family, but to some degree, this is mental health and nothing to do with the AI. You have to be mentally impaired to believe the things it’s telling you are truth.
Every time I read one of these stories, I’m reminded of things from my timeline that had nothing to do with AI. A man killed himself after watching the Avatar movie because their world was so beautiful he just couldn’t stand living on earth anymore and thought he’d be reincarnated as a navi. Kids jumping off roofs after watching Pokémon or Dragon Ball Z thinking they can really fly.
Should those forms of media not exist because a handful of people can’t cope with reality? No. I do believe AI is a scourge on humanity. But I see this as a major mental health catastrophe that happened. He was not stable to begin with. Any sane or normal person would not think this was real.
Wouldn’t be the father’s fault? I mean if he has a son that is mentally challenged, isn’t his responsibility to not let him use AI or monitor it.