Your New Laptop Has a Superchip You Don't Use

New laptops are now commonly featuring Neural Processing Units (NPUs), specialized chips designed to run AI workloads with significantly lower power consumption than CPUs or GPUs. These NPUs consume around 2W, a stark contrast to the over 100W standard discrete graphics require for similar tasks. Advisory firm Gartner indicates that AI-capable PCs now constitute over half of global shipments, driven by Microsoft's 2024 requirement for new laptops to achieve 40 TOPS of dedicated NPU performance to access Windows AI features.

NPUs excel at inference, the phase of running AI models, by efficiently executing matrix multiplication and nonlinear activation functions. This allows for features like background blur on video calls to operate continuously without draining battery life or generating excessive heat. However, NPUs are not suitable for training large AI models or running cloud-based services like ChatGPT. Their current practical applications are limited to low-power, always-on tasks such as webcam effects, noise suppression, and real-time translation.

The push for NPUs is largely due to Microsoft's Copilot+ PC certification, which mandates a 40 TOPS NPU, 16GB RAM, and 256GB storage. This requirement, focusing specifically on NPU performance rather than GPU capabilities, has led manufacturers to integrate these chips. Leading chipmakers like Qualcomm, Intel, and AMD have released processors with NPUs exceeding Microsoft's minimum, with Qualcomm's latest offering reaching up to 85 TOPS. Despite this hardware advancement, software that leverages NPUs remains scarce, with only a handful of applications currently utilizing them effectively, such as Windows Studio Effects and Teams noise suppression.