What are Recursive Language Models (RLMs) developed by MIT?

RLMs are an inference technique from MIT CSAIL that allows LLMs to process extremely long prompts by treating them as external environments they can code against.

How do Recursive Language Models overcome LLM context limitations?

RLMs enable LLMs to write code to inspect and process text snippets, rather than requiring the entire prompt to fit into the context window, thus handling millions of tokens.

What are the performance benefits of MIT's RLMs?

RLMs show significant performance gains on long-context tasks, outperforming standard models dramatically on benchmarks with millions of tokens, including complex reasoning and code understanding.

Home / Technology / MIT's RLMs Unlock Millions of Tokens

MIT's RLMs Unlock Millions of Tokens

21 Jan

•

Summary

RLMs treat prompts as external environments for LLMs to code against.
The technique processes millions of tokens without retraining models.
RLMs show significant performance gains on large-scale benchmarks.

Researchers at MIT CSAIL have introduced Recursive Language Models (RLMs), a novel inference technique that redefines how large language models (LLMs) handle extensive prompts. Instead of fitting entire texts into a model's context window, RLMs enable LLMs to programmatically interact with prompts as external environments. This approach allows models to decompose and recursively process text snippets, effectively reasoning over millions of tokens without the need for retraining.

This framework reframes long-context reasoning as a systems problem, offering enterprises a viable solution for complex tasks such as codebase analysis and legal review. By acting as a wrapper around existing models, RLMs can be seamlessly integrated into current applications. The method draws inspiration from classical computing's 'out-of-core' algorithms, loading text as a variable that the LLM then manipulates using code to extract and analyze relevant chunks.