AI Agents Learn Faster: New SIFT Framework Cuts Costs

Researchers from MIT and Sakana AI have introduced SIFT (Recursive Self-Improvement via Fast Tree Search), a novel framework designed to make the improvement of coding agents more cost-effective. Traditional methods require extensive testing for each potential modification, leading to significant computational expense. SIFT addresses this by integrating a language model as a judge to compare candidate agents prior to full benchmark evaluations. This intermediary step helps filter out less promising changes, saving time and resources. The framework also supports asynchronous processing, allowing new modifications to be generated and evaluated while earlier ones are still undergoing extensive testing. This parallel approach accelerates the exploration of different improvement pathways. SIFT achieved notable performance on the Polyglot coding benchmark, reaching 35.1% accuracy in under five hours of wall-clock time, with associated costs around $150 in API credits and 42 CPU hours. This demonstrates a more selective approach to spending evaluation resources for teams optimizing AI coding harnesses for specific tasks. The system's ability to propose and test changes concurrently mirrors continuous integration pipelines, enabling faster iteration and development cycles.