Schulen nutzen mittlerweile KI, um die psychische Gesundheit von Kindern auszuspionieren

(Repost to fix title)

Not sure how long this has been around, but I just learned this was a thing.

Call me paranoid, but it’s only a matter of time before this gets horribly abused. Sure, now it’s (arguably) for a good cause. Although I wonder how effective it even is for suicide prevention, it sure didn’t help the girl in the article much, she just got smarter about it.

But who gets to decide what gets flagged, why it gets flagged, and the repercussions of that? For now it’s something everyone can get behind – mental health, but did anyone ask for this? Did anyone sign up for it? It sounds like they just arbitrarily made the decision to enact this. So then what’s to stop them from arbitrarily adding something else to monitor – Drugs, porn, and then more.. Alerting abusive parents to their teen’s social life as they try to form a life away from them, in the guise of „protecting them from grooming“? It could even undermine attempts to move away from an unsafe environment because those same keywords could be triggered to prevent some sort of abduction or something.

Not to mention how icky it is to me that some corporation is saving all this mental health info about a large number of children. Sounds like somehow they’re skirting around HIPPA.