Home / Technology / AI Models Shrink Dramatically for Your PC
AI Models Shrink Dramatically for Your PC
18 Sep
Summary
- PrismML makes reasoning LLMs small enough for PCs and smartphones.
- Bonsai 2 27B model is 9x to 10x smaller than the original.
- Compressed models retain nearly all original performance benchmarks.
AI lab PrismML is revolutionizing large language models (LLMs) by making them exceptionally small without sacrificing performance. Founded by Caltech researchers, the startup aims to enable capable, reasoning LLMs to fit on personal computers and smartphones. Their latest release, Bonsai 2 27B, compresses Alibaba's Qwen3.8 27B model to just 5.9 GB, a remarkable 9x to 10x reduction in memory usage.
This compression technique, which simplifies model 'weights' to three states (0, +1, -1), allows the Bonsai 2 model to match 98% of the original Qwen's benchmark scores. Previous versions also saw significant adoption, with over 11 million downloads of an earlier model.
PrismML plans to apply this technology to even larger models, expecting the compression process to become easier with increased size. This innovation promises to bring advanced AI capabilities directly to user devices, offering enhanced privacy and accessibility.
