AI Safety Bill: 50 Deaths or $1B Damage Triggers Halt

Anthropic has put forth a draft bill aimed at regulating artificial intelligence, proposing specific conditions under which AI model releases could be halted. This legislation, intended for consideration by Congress, outlines scenarios of "foreseeable, imminent risk" that could trigger a block on distribution. Such risks are defined as those involving potential fatalities exceeding 50 people or property damage surpassing one billion dollars. These catastrophic risks must also originate from specific causes, including the creation of a biological weapon, the AI model evading developer controls, or a significant cyberattack.

This proposed framework suggests the utilization of "independent evaluators" registered with the U.S. Commerce Department to assess the potential for disaster posed by AI models. While these evaluators might be compensated by the AI labs, their payment must not be contingent on the outcome of their evaluations. The bill mandates that if evaluators determine an imminent catastrophic risk, they must notify the Commerce Department within 72 hours. Subsequently, the Commerce Department would need to seek a judicial order from a D.C. judge to prevent the model's release, explicitly forbidding federal agencies from unilaterally halting distribution.