Mac AI Boost: Ollama Integrates MLX

Ollama, a system for running large language models locally, has introduced support for Apple's open-source MLX framework. This integration is specifically designed to enhance performance on Macs equipped with Apple Silicon chips (M1 or later).

Alongside MLX support, Ollama has optimized its caching mechanisms and incorporated Nvidia's NVFP4 format for model compression. These advancements collectively lead to more efficient memory usage and faster processing, particularly benefiting users who run AI models on their personal computers.

The timing of these improvements aligns with a growing interest in local large language models. As developers encounter limitations with cloud-based services, such as rate limits and subscription costs, the appeal of running models on local hardware has surged.

This new support is currently in preview within Ollama version 0.19 and initially supports a specific 35-billion-parameter model from Alibaba. Hardware requirements are substantial, mandating at least 32GB of RAM on an Apple Silicon Mac.

Ollama is also leveraging the Neural Accelerators found in newer Apple GPUs, which should further boost speed metrics like tokens-per-second. While local models still trail frontier models in benchmarks, they are becoming capable enough for tasks previously requiring paid subscriptions, offering distinct privacy advantages.

Apple's MLX framework optimizes access to the unified memory architecture on its chips, differing from previous Ollama targets that relied on dedicated GPUs. This evolution represents a positive step for modern Mac users, potentially narrowing the performance gap with cloud-based models.