Inkling-Small: Powerful AI Now 75% Smaller, Cheaper

Thinking Machines recently unveiled Inkling-Small, a highly capable AI language model that closely matches the performance of its larger counterpart, Inkling. This new model boasts 276 billion total parameters and 12 billion active parameters, significantly reducing compute requirements and inference costs. Inkling-Small accepts multimodal inputs including text, images, and audio, supporting a context window of up to one million tokens. Its permissive Apache 2.0 license encourages widespread commercial adoption and modification by enterprises seeking flexibility.

Despite its smaller size, Inkling-Small demonstrates competitive performance across various benchmarks, even surpassing the larger Inkling in some coding and reasoning tasks. The model's architecture, a sparse Mixture-of-Experts, enables efficient processing by activating only a fraction of its parameters per token. While not suitable for laptops, Inkling-Small offers enterprises a more manageable option for self-hosting compared to its predecessor, lowering deployment barriers.

The release signifies Thinking Machines' growing ability to develop and deploy AI models more routinely. Inkling-Small's availability on Hugging Face, alongside fine-tuning support via the Tinker API, provides developers with multiple deployment pathways. This strategic release offers businesses a compelling balance of performance, cost, and licensing flexibility in the open-source AI landscape.