Startup Vals Revolutionizes AI Model Testing

Vals, a startup established in 2024, is addressing critical flaws in the AI industry's model benchmarking practices. Traditional benchmarks, often not designed for modern AI capabilities, are being circumvented by companies seeking to highlight their successes. Vals aims to create a more robust and accurate evaluation system.

The company recently announced a $40 million Series A funding round, led by Andreessen Horowitz, following a successful seed round. This investment underscores the growing demand for sophisticated AI model validation.

Vals differentiates itself by not publicly disclosing its test materials, preventing companies from training models specifically to pass these evaluations. Instead, Vals focuses on assessing AI models' performance on complex, industry-specific tasks, evaluating their real-world impact and potential negative consequences.

Expanding beyond traditional sectors, Vals is developing benchmarks for areas such as recursive self-improvement, mental health, cybersecurity, biosecurity, and even the application of international humanitarian law. Companies pay Vals for these evaluations, which help them identify areas for improvement and build trust.

Vals reported an eight-fold increase in revenue compared to last year and has expanded its team from eight to 25 employees. The startup plans further growth, including relocating to a larger office and adding more staff. They have also launched a program to provide model evaluations to federal agencies. Krishnan believes their benchmarking system will be pivotal as AI companies go public and become more integrated into the global economy.