Waymo's AI: Safety First in Self-Driving

Waymo, Alphabet's autonomous vehicle company, employs rigorous methods to ensure the safety of its AI systems that navigate real-world environments. A core strategy is 'eval-centric development,' embedding continuous evaluation into the engineering process rather than treating it as a final step before deployment. This approach, which has guided Waymo through over 220 million autonomous miles with superior safety records compared to human drivers, is adaptable for enterprises using AI in various sectors.

The company stresses that evaluations must persist beyond initial launch, encompassing driving, simulation, and validation. This ongoing assessment is vital as models, business processes, and user behaviors evolve. Waymo also scrutinizes datasets and metrics, focusing on rare and dangerous scenarios to preempt potential financial, legal, or reputational damage, a principle applicable to all AI applications.

Human oversight is integral to Waymo's release decisions, with safety leaders approving software and expansions, countering the notion of fully automated AI. Despite growing resource demands, Waymo pursues efficiency in its AI infrastructure and model training, emphasizing data efficiency. The company utilizes advanced AI models, including generative multimodal models, for both onboard vehicle systems and off-board development, necessitating optimization for real-time inference and supporting infrastructure.

Internally, Waymo uses AI agents as productivity tools, but subject them to evaluation to ensure accuracy and trustworthiness. The overarching lesson for enterprises is that successful agentic AI requires defined objectives, representative data, continuous testing, efficient infrastructure, and accountable human decision-makers, ultimately aiming to earn user trust.