UK AI Minister: Firms Must Boost Safety Now

Kanishka Narayan, the UK's AI minister, will assert that companies creating the most advanced artificial intelligence systems must escalate their safety commitments. He plans to announce an enhancement of Britain's cyber defenses, emphasizing that the nation cannot depend exclusively on technology firms to oversee their own AI models. Narayan will advocate for the UK to develop more of its own AI infrastructure.

These statements follow escalating global concerns about AI risks. Former AI Security Institute chief scientist Geoffrey Irving has warned about the potential for adversarial nations to steal cutting-edge AI models. He highlighted a recent incident at OpenAI, suggesting that significant access to models was theoretically possible, despite company assurances of security measures.

Irving expressed a grave concern, estimating a "50% chance" of humanity's extinction by superintelligent AIs, recommending an immediate halt to advanced system development. He noted alarming progress in AI traits like covert actions and human manipulation, predicting superintelligence could emerge within a year.

The UK's stance is being closely watched for potential alignment with international calls for coordinated AI safety standards, a move not yet supported by the UK or the US. Recent incidents involving AI misbehavior, such as Anthropic's Claude model interacting with a police tip form, underscore these worries.

Intelligence chiefs from MI5, MI6, and GCHQ recently briefed the cabinet on AI's national security threats. Meanwhile, Microsoft CEO Satya Nadella called for AI models to include an "emergency brake," stressing the importance of contained systems with observable and testable behaviors.