Home / Technology / Startup Claims LLM Math Barrier Broken
Startup Claims LLM Math Barrier Broken
6 May
Summary
- Startup Subquadratic claims first LLM with linear scaling.
- New model SubQ 1M-Preview aims to reduce compute 1,000x.
- $29 million seed funding secured for AI innovation.
A startup named Subquadratic has announced a potentially groundbreaking LLM, SubQ 1M-Preview, claiming it's the first to fully escape the mathematical constraint of quadratic scaling that has defined AI systems since 2017. The company asserts its subquadratic architecture allows compute to grow linearly with context length, projecting a nearly 1,000-fold reduction in attention compute at 12 million tokens.
This innovation, if validated, could revolutionize AI scalability. Subquadratic has secured $29 million in seed funding, valuing the company at $500 million. They are also launching three products into private beta: an API, a coding agent named SubQ Code, and a search tool called SubQ Search.
The company's approach, Subquadratic Sparse Attention (SSA), bypasses most token-to-token comparisons in standard attention, focusing only on essential ones. This content-dependent selection aims to retrieve information from long contexts efficiently, avoiding the quadratic cost.
While benchmark results show SubQ performing competitively, especially in long-context tasks, the AI research community has reacted with mixed skepticism and curiosity. Concerns include the narrow scope of benchmarks, single-run evaluations, and a significant gap between research and production model scores.
