Home / Technology / Marvell's Light-Speed Memory: A New Era for AI?
Marvell's Light-Speed Memory: A New Era for AI?
24 Oct
Summary
- Marvell's Photonic Fabric Memory Module uses light for shared memory.
- Appliance offers 32TB of DDR5 and 1TB HBM3e for multiple hosts.
- Customer sampling for the memory solution is targeted for 2027.
Marvell has introduced its Photonic Fabric Memory Module, a novel solution designed to address the escalating memory demands of AI infrastructure. This module utilizes optical interconnects, allowing for high-capacity, high-bandwidth memory to be shared across multiple hosts, thereby reducing the reliance on GPUs acting solely as memory controllers.
The Photonic Fabric Memory Appliance, a configuration of 16 modules, offers a pooled 32TB of DDR5 memory and 1,152GB of HBM3e. This architecture allows for up to 16 hosts to share this substantial memory resource. Marvell has slated customer sampling for this pre-silicon design to commence in 2027.
This technology aims to tackle challenges like the rapidly growing KV cache in large language models. By enabling a cross-node KV cache, accelerators can reuse context data, improving performance and efficiency. This modular approach suggests memory can scale more independently of compute power, potentially offering significant capital expenditure savings.
While Marvell's solution is still in development, competitors like Nvidia are also advancing memory solutions. Nvidia has invested in Marvell and is developing its own technologies, including BlueField-4 STX and co-packaged optics, indicating a broader industry trend toward independent memory scaling.
