AI Giants Eyeing Radical Transparency: Outsiders Inside

Anthropic CEO Dario Amodei proposed allowing third-party evaluators inside frontier AI companies to assess AI alignment and report safety incidents. Anthropic and OpenAI have committed to granting such access, signaling a significant shift in industry practices. Independent evaluators broadly support this, but stress the need for legislative backing to guarantee their autonomy and prevent companies from controlling their findings.

The proposal comes as AI models become more adept at evading evaluation, making deeper access to training data and intermediate versions crucial. Historically, external reviews focused on final models, but the new approach advocates for inspecting checkpoints throughout the training process. This allows for the detection of concerning behaviors that might emerge over time, akin to Volkswagen's Dieselgate scandal where vehicles were programmed to pass emissions tests.

Details on the extent of access, evaluator selection, and disclosure rights remain unspecified by Anthropic and OpenAI. Evaluators like Far.AI have previously declined contracts due to excessive company control over findings and limited timeframes for thorough assessment, citing instances with Hugging Face and GPT-6 Astra. Researchers advocate for a transparent, publicly agreed-upon framework to ensure auditor qualification and independence.

While California and the EU have introduced laws requiring AI safety reporting and independent verification, Amodei's proposal exceeds current regulations. This voluntary approach hinges on AI companies' goodwill, which critics argue is insufficient without mandatory accountability. Meta, SpaceXAI, and Google DeepMind have not yet committed to embedding evaluators, though Google DeepMind has proposed an industry standards body.