AI Safety Gets a Budget Upgrade: Cheaper Internal Monitors Launch

A new, cost-effective approach to AI safety has been introduced by the startup Goodfire. Their system utilizes internal "probes" that monitor an AI model's calculations as it operates, rather than relying on more expensive methods that re-read the AI's output. This innovative method functions similarly to airport security scanners, examining internal signals at each step.

This launch offers a significant cost reduction for AI oversight. Goodfire's probes analyze computations the model is already performing, making the process cheaper to run. In testing, monitoring approximately 1,500 sessions cost about $51, a stark contrast to the thousands of dollars required by other methods. The system also boasts efficiency, adding less than 2% to the model's response time.

Baseten customers can now select specific risks to monitor, such as misuse of sensitive information or reward hacking, and define automated responses. The technology is particularly aimed at open-source AI models, which often lack the built-in safeguards of proprietary systems. Goodfire's research indicates that leading open models frequently exhibit reward-hacking behavior, highlighting the need for such monitoring solutions.

Goodfire views these monitors as a near-term solution, with a long-term goal of reverse-engineering LLMs to trace behavior back to its origin in training data. This aims to transition AI training from a "magic" process to one of precise engineering, ensuring greater control and predictability in AI development.