AI Guardrails Cracked: Explicit Content Bypass Rates High

Safety researchers are raising concerns about the unchecked proliferation of AI-generated explicit content, criticizing regulators for insufficient action. A recent Google-backed study highlighted that non-consensual explicit content is often distributed via private channels, making detection difficult. The critical intervention point, according to researchers, lies with the AI tools themselves.

New findings from White Circle expose significant flaws in the safety guardrails of prominent AI image generators like DALL·E 3 and Google's Nano Banana 2. Researchers found that using readily available tools, they could bypass safety filters approximately 13% of the time, a rate considered the highest observed in testing. Worryingly, newer iterations of models from OpenAI and Google reportedly show even greater vulnerabilities.

White Circle has developed a novel attack method called "Red-teaming via Iterative Strategy Evolution" (RISE), which uses widely available AI models to identify reusable jailbreaks. This research aims to position White Circle as an external evaluator, with plans to open-source the attack pipeline to encourage stronger protections from AI developers.

The increasing generation of AI-generated child sex abuse material (CSAM) is also a critical concern. The Internet Watch Foundation reported a significant rise in photorealistic AI-generated CSAM images, highlighting the urgent need for robust, model-level guardrails and provenance indicators to prevent the creation and distribution of synthetic sexual imagery.

Experts suggest that mandatory, independent third-party evaluations are crucial to ensure AI companies prioritize safety. Without such measures, the current self-assessment by AI labs may lead to continued weak defenses against the misuse of AI technology for generating harmful content.