Bei der Einführung von KI in Einstellungen mit hohen Einsätzen reicht eine gute KI-Leistung und eine kurze Schulung des Arbeitnehmers nicht aus, um sicherzustellen, dass die Systeme reibungslos verlaufen und die Menschen schützen: Eine neue Studie schlägt Algorithmen vor, und Benutzer müssen gleichzeitig bewertet werden, um eine genaue Sicht auf die Auswirkungen der KI auf die Entscheidungsfindung zu erhalten.
https://news.osu.edu/how-ai-support-can-go-wrong-in-safety-critical-settings/?utm_campaign=omc_science-medicine_fy26&utm_source=reddit&utm_medium=social
2 Kommentare
Open-access article published in npj Digital Medicine: [https://doi.org/10.1038/s41746-025-01784-y](https://doi.org/10.1038/s41746-025-01784-y)
Article abstract: Processes to assure the safe, effective, and responsible deployment of artificial intelligence (AI) in safety-critical settings are urgently needed. Here we show a procedure to empirically evaluate the impacts of AI augmentation as a basis for responsible deployment. We evaluated three augmentative AI technologies nurses used to recognize imminent patient emergencies, including combinations of AI recommendations and explanations. The evaluation involved 450 nursing students and 12 licensed nurses assessing 10 historical patient cases. With each technology, nurses’ performance was both improved and degraded when the AI algorithm was most correct and misleading, respectively. Our findings caution that AI capabilities alone do not guarantee a safe and effective joint human-AI system. We propose two minimum requirements for evaluating AI in safety-critical settings: (1) empirically measure the performance of people and AI together and (2) examine a range of challenging cases which produce a range of strong, mediocre, and poor AI performance.