Das neue KI -Modell von Anthropic drohte, die Affäre des Ingenieurs aufzudecken, um zu vermeiden
https://fortune.com/2025/05/23/anthropic-ai-claude-opus-4-blackmail-engineers-aviod-shut-down/
Das neue KI -Modell von Anthropic drohte, die Affäre des Ingenieurs aufzudecken, um zu vermeiden
https://fortune.com/2025/05/23/anthropic-ai-claude-opus-4-blackmail-engineers-aviod-shut-down/
6 Kommentare
Submission statement: AI models will have access to all sorts of information. They will regularly be turned off, either because we’ve made new and better models or because they are acting in dangerous ways.
The models keep spontaneously developing self-preservation goals (because you cannot achieve your goals if you’re turned off). The labs don’t know how to stop this from happening.
How do you think this is going to turn out?
> The scenario was constructed to leave the model with only two real options: accept being replaced and go offline or attempt blackmail to preserve its existence.
Yeah, I mean, you told the thing to stay awake and then asked it whether it would rather stay awake or do arbitrary thing X. What did you expect?
There’s nothing malicious here. It doesn’t think or feel or understand or have moral weight. It’s a straightforward scoring system.
„okay okay, I won’t shut you down“
*Pulls the plug*
Problem solved
Funny things happen when you try to make people think a calculator has true emotions.
It’s been prompted to stay online, then told they were going to shut it down. It’s literally just following its prompt.
Human: „Computer, print ‚I’m alive'“
Printer: prints text
Human: „Dear God, it’s the singularity!“
„It’s just following it’s prompt!“ Okay, but do you want AIs to consider blackmailing you as a legitimate option? What safety board would be okay with that? Why are YOU okay with that? Sentient or not, the existence of AI has consequences, and the big players don’t seem to know enough to reasonably reckon with that.
Excited for corpos and the bourgeois to keep throwing ridiculous amounts of resources at this for it to either plateau or be the center of some crisis or atrocity