MIT's RLMs Unlock Millions of Tokens

Researchers at MIT CSAIL have introduced Recursive Language Models (RLMs), a novel inference technique that redefines how large language models (LLMs) handle extensive prompts. Instead of fitting entire texts into a model's context window, RLMs enable LLMs to programmatically interact with prompts as external environments. This approach allows models to decompose and recursively process text snippets, effectively reasoning over millions of tokens without the need for retraining.

This framework reframes long-context reasoning as a systems problem, offering enterprises a viable solution for complex tasks such as codebase analysis and legal review. By acting as a wrapper around existing models, RLMs can be seamlessly integrated into current applications. The method draws inspiration from classical computing's 'out-of-core' algorithms, loading text as a variable that the LLM then manipulates using code to extract and analyze relevant chunks.

Experiments validating RLMs demonstrated substantial improvements, particularly at the 10 million+ token scale. On benchmarks like BrowseComp-Plus, where standard models achieved 0%, an RLM powered by GPT-5 reached 91.33%. The framework also excelled in computationally intensive tasks, achieving an F1 score of 58% on OOLONG-Pairs, a feat that paralyzed base GPT-5 models. Despite workflow complexity, RLMs often maintained comparable or lower costs, though outlier runs can increase expenses.