AI Nudes: Tech Giants' Role in Deepfake Crisis

The proliferation of AI-generated non-consensual intimate imagery (NCII) and child sexual abuse material (CSAM) presents a significant online threat. Generative AI allows for the creation of such content at an unprecedented scale, transforming how it is distributed and removed. Experts highlight the deepfake nude crisis, with reports indicating widespread impact on schools and students globally.

While victims suffer severe psychological distress, attention is increasingly turning to the tech companies facilitating this content. Critics argue that faulty moderation systems and advertisements inadvertently empower this abuse. Companies like Meta assert their commitment to combating deepfakes, pointing to strict rules against related apps and legal actions against operators.

Industry-wide collaborations, such as the Tech Coalition's Lantern system, aim to disrupt repeat offenders by sharing 'signals' across platforms. Since 2023, Lantern has facilitated millions of signals, leading to hundreds of thousands of enforcement actions, including account bans and content removal. This cross-platform intelligence sharing is seen as crucial for addressing abuse that spans multiple services.

Despite these efforts, researchers have identified numerous advertisements promoting apps capable of creating deepfake nudes, some linked to CSAM. Critics argue that tech giants, with vast resources, are incentivized to overlook these issues, placing the blame on individuals instead. Companies maintain they are actively removing harmful ads and pursuing legal avenues, emphasizing the need for rapid adaptation to evolving threats.