AI Labs Replaced by Open Scientific Method

A new nonprofit, Trillium Labs, is challenging the prevailing secrecy in frontier AI research. Founded by scientists Nathan Lambert and Tom Zick, the organization advocates for an open scientific method, publishing experimental details to allow broader community scrutiny and replication.

Lambert argues that the current closed approach by major AI labs hinders the ability to scrutinize ideas and develop new mitigation strategies for AI risks. He believes this secrecy is a step backward from the established scientific method, which has historically helped humanity build better futures.

Trillium Labs will focus on post-training fine-tuning of large models and recursive self-improvement (RSI), a process where AI contributes to further research. The potential for RSI to lead to a loss of human control has alarmed many researchers, with recent warnings highlighting it as an existential threat.

The nonprofit also plans to explore how reinforcement learning shapes AI model behavior. By publishing details of these experiments, Trillium Labs aims to foster surprising insights as outside researchers analyze the work, contributing nuance to the critical global discussion on AI development and safety.