Home / Technology / AI Fights Child Exploitation with New Detection Tool

AI Fights Child Exploitation with New Detection Tool

Summary

  • New AI classifier examines visual CSAM signs for detection.
  • ChatGPT now issues warnings and 24-hour suspensions for abuse attempts.
  • Tool offers free access to vetted organizations for child protection.
AI Fights Child Exploitation with New Detection Tool

OpenAI is enhancing its child safety measures with a new AI classifier designed to detect child sexual abuse material (CSAM), including AI-generated content. This tool examines visual characteristics rather than relying solely on digital fingerprints, enabling the identification of novel abusive material. The system assigns scores to suspected content, prioritizing cases for human review. OpenAI developed this technology in collaboration with the National Center for Missing & Exploited Children (NCMEC) and is providing it at no cost to vetted organizations via its API.

Early testing indicates the new classifier is promising, flagging fewer false positives while maintaining detection rates. For instance, in evaluations, it flagged approximately 48% fewer pictures for review compared to existing protections, with flagged images being twice as likely to contain abuse. Collaborating organizations reported significant confirmation rates of previously unidentified CSAM and effective detection of animated child sexualization.

In parallel, OpenAI is strengthening ChatGPT's safeguards. Users attempting to access or generate CSAM will now encounter new warnings explaining the harm of child sexualization and AI-generated abuse. Following a warning, the conversation will be terminated, and access to ChatGPT will be suspended for 24 hours. Repeated offenses could lead to longer restrictions or account closure.

OpenAI is also expanding its support for child exploitation investigations through programs like the Tech Coalition's Lantern and by developing specialized AI tools for law enforcement. These tools assist investigators in reviewing chat logs and organizing timelines from legal records. Guidance is also being published for developers on detecting suspected abuse and establishing content review procedures.

Parents are advised to discuss AI-generated fakes with children, review social media privacy settings, and report suspected abusive material to NCMEC's CyberTipline. NCMEC also offers a 'Take It Down' service to help limit the spread of explicit images involving minors by creating digital fingerprints.

NCMEC reported over 400,000 reports involving generative AI in 2025. OpenAI's initiatives are seen as positive steps in combating online child exploitation, with hopes that other AI companies will follow suit.

Disclaimer: This story has been auto-aggregated and auto-summarised by a computer program. This story has not been edited or created by the Feedzop team.

Read more news on

Property Code: 5571