Der Aufstieg gefälschter Pornografie in Schulen: „Ein Mädchen war so entsetzt, dass sie sich übergeben musste“ | Die Verwendung von „Nudify“-Apps wird immer häufiger eingesetzt, und Hunderte von Lehrern haben Bilder gesehen, die von Schülern, oft von Gleichaltrigen, erstellt wurden. Die Auswirkungen sind enorm – und nehmen schnell zu

Some of the concerning issues raised here:

>‘It worries me that it’s so normalised. He obviously wasn’t hiding it. He didn’t feel this was something he shouldn’t be doing. It was in the open and people saw it. That’s what was quite shocking.”
>
>A headteacher is describing how a teenage boy, sitting on a bus on his way home from school, casually pulled out his phone, selected a picture from social media of a girl at a neighbouring school and used a “nudifying” app to doctor her image.
>
>…
>
>The boy on the bus is just one example of how deepfakes and easily accessed nudifying technology are playing out among schoolchildren – often to devastating effect. In Spain last year, 15 boys in the south-western region of Extremadura were sentenced to a year’s probation after being convicted of using AI to produce fake naked images of their female schoolmates, which they shared on WhatsApp groups. About 20 girls were affected, most of them aged 14, while the youngest was 11.
>
>In Australia, about 50 high school students at Bacchus Marsh grammar in Victoria reported that their images had been faked and distributed – the mother of one student said her daughter was so horrified by the sexually explicit images that she vomited. In the US, more than 30 female students at Westfield high school in New Jersey discovered that deepfake pornographic images of them had been shared among their male classmates on Snapchat.
>
>It’s happening in the UK, too. A new poll of 4,300 secondary school teachers in England, carried out by Teacher Tapp on behalf of the Guardian, found that about one in 10 were aware of students at their school creating “deepfake, sexually explicit videos” in the last academic year.
>
>Three-quarters of these incidents involved children aged 14 or younger, while one in 10 incidents involved 11-year-olds, and 3% were younger still, illustrating just how easy the technology is to access and use. Among participating teachers, 7% said they were aware of a single incident, and 1% said it had happened twice, while a similar proportion said it had happened three times or more in the last academic year.
>
>…
>
>Laura Bates, the founder of the Everyday Sexism Project, says there is something particularly shocking about deepfake images. In her book The New Age of Sexism: How the AI Revolution Is Reinventing Misogyny, she writes: “Of all the forms of abuse I receive they are the ones that hurt most deeply – the ones that stay with me. It’s hard to describe why, except to say that it feels like you. It feels like someone has taken you and done something to you and there is nothing you can do about it. Watching a video of yourself being violated without your consent is an almost out-of-body experience.”
>
>Among school-age children, the impact can be huge. Girls and young women are left feeling violated and humiliated. School friendship groups are shattered and there can be a deep sense of betrayal when one student discovers another has created a deepfake sexualised image of them, and shared it around the school. Girls can’t face lessons, while teachers with little training do their best to support and educate. Meanwhile, boys and young men are being drawn into criminal behaviour, often because they don’t understand the consequences of their actions.
>
>…
>
>“For everyone working in schools, it feels like new sets of challenges and risks are constantly being thrown up by technological developments,” he says. “AI generally – and particularly deepfakes and nudify apps – feel like the next train coming down the track.
>
>“‘More education’ is appealing as a solution to these sorts of challenges – because it’s intuitive and relatively easy – but on its own it’s like trying to hold back a forest fire with a water pistol. And likewise, the police seem completely overwhelmed by the scale of these issues. As a society we need broader solutions, and better strategy.”
>
>He continues: “We should all try to imagine how we would have felt 20 years ago if someone had suggested inventing a handheld device which could be used to create realistic pornographic material that featured actual people that you know in real life. And then they’d suggested giving one of these devices to all of our children. Because that’s basically where we are now. We’re letting these things become ‘normal’ on our watch.”
>
>…
>
>“The way that young people are using these technologies is not necessarily all bad,” she says, “but what they need is better media literacy.” She welcomes the government’s updated relationships, sex and health education guidance, which “recognised that misogyny is a problem that needs to be tackled in the school system”. However, she says: “They need to put the dots together. They need to join up a concern with gender and sexual-based violence with technology. You can’t rely on Ofcom or the regulators to protect young people. We need proactive, preventive education.”
>
>Where is the government in all this? “Our new relationships, sex and health education guidance will make sure that all young people understand healthy relationships, sexual ethics and the dangers of online content such as pornography and deepfake,” a Department for Education spokesperson said. “As part of our Plan for Change mission to halve violence against women and girls, we are also providing schools with new funded resources to help teachers explain the law and harms relating to online content as part of their age-appropriate lessons.”
>
>Ringrose stresses the urgency. “These issues are happening – non-consensual creation and distribution of images is happening. These technologies are at people’s fingertips. I mean, it’s super-easy for any kid to access these things.”

It’s been clear for a while with the advent of these technologies that industry is unwilling to implement meaningful guardrails of their own accord, and we’re seeing some of the consequences of this inaction and the lag by regulators in this space as well. Clear and effective regulation with attendant enforcement and meaningful penalties, along with adequate resources to educators, parents, and students is critical here if we are serious about mitigating the impacts of this technology on our society.

This is also a necessary reminder that there are inherent costs to new technologies that aren’t necessarily borne by the creators or the users, and that those creating these systems should be well aware of this from the start.

Obvious this would happen, anyone who worked on this technology and thought this wouldn’t happen is oblivious to the consequences of their actions 

How is this any different than photo shopping someone’s face onto a porn stars body?

Edit: God forbid someone asks a fucking question Jesus Christ

I thought NSFW apps were banned on most major platforms?

This technology should be outlawed. Voice and image replication technology of real people isnt good for anything but problems.

As someone who is getting a little older, there is no excuse for not having been ahead of this. People saw this coming when text messaging first started, and I’m not exaggerating. It is entirely pathetic that this kind of thing wasn’t preempted.

This is obviously not ok and we should do what we can to control it, but limiting access or making laws against it is just another case of trimming branches off of a tree with rotten roots. Treating symptoms not causes. Is this not the logical result of misogyny, objectification, and hyper-sexualization in our culture? Women, girls, and anyone in general should not be objectified. Nudity should not be inherently sexual. Sexuality should not be shameful. I think western civilization is well past the point of meaningfully addressing any issues like these, but if we’re not going to have the conversations then we’re evidently ok with barreling down the path that will continue to produce more and more shit like this because we’re not willing to admit that blindly pursuing and rewarding technological innovation without considering the worst ways it could be wielded might have horrific unintended consequences beyond our control.

Creators of those apps should first of all be rooted out and penalized severely, because they know what kind of trauma they’re causing society.

People can say it’s no big deal, but YOU don’t get to predetermine what traumatizes other people. This especially harms young people that are very emotionally vulnerable.

Father of young kids here. Can we as parents all agree, no personal screens until adulthood? There are a few in my community who are on board, but we need more.

I could only imagine the chaos this would have caused if it was available when I was a little fella still in school…