Home / Technology / AI Learns to Cut LLM Costs by 70%
AI Learns to Cut LLM Costs by 70%
29 May
Summary
- AutoTTS framework automates LLM compute strategy discovery.
- AI-designed strategies reduce token consumption by up to 69.5%.
- Discovery process costs only $39.90 and takes 160 minutes.
Test-time scaling (TTS) improves large language model (LLM) performance by allocating extra compute cycles during inference. Traditionally, TTS strategies have been manually crafted, relying on human intuition. Researchers from Meta, Google, and universities have developed AutoTTS, a framework that automates the discovery of optimal TTS strategies.
AutoTTS reframes strategy design as an algorithmic search problem. An explorer LLM iteratively proposes and refines TTS controllers, which dictate how AI models allocate computational budgets. This process utilizes an offline replay environment with pre-collected reasoning trajectories to make the search computationally affordable.
Experiments using AutoTTS on Qwen models demonstrated significant cost savings and performance gains. One AI-discovered controller reduced token consumption by approximately 69.5% compared to manual methods while maintaining accuracy. On the GPQA-Diamond benchmark, inference token costs were slashed from 510K to 151K tokens.
This automated approach offers two main operational benefits for enterprises: it can raise peak performance by intelligently reallocating compute budgets and enables cost-effective custom development. The entire discovery process for AutoTTS cost only $39.90 and took 160 minutes, making tailored LLM optimization strategies more attainable.
