Home / Technology / Desktop AI Runs 100B Models Locally
Desktop AI Runs 100B Models Locally
2 Aug
Summary
- Acra's Agent Box runs 100B parameter AI models on local hardware.
- Proprietary G≡LIX 1 chip offers 700 TOPS processing for AI.
- System offers significant cost and power savings over cloud.
Singaporean startup Acrab has launched the Agent Box, a compact desktop computer designed to run large language models with up to 100 billion parameters using only local hardware. The device is powered by Acrab's custom G≡LIX 1 chip, a 5nm processor engineered for efficient edge computing and delivering a remarkable 700 TOPS of performance. This innovation promises substantial cost reductions and halved power consumption compared to existing cloud-based AI solutions.
The G≡LIX 1 chip integrates a 20-core Arm CPU, a 3 TFLOPS GPU, and a specialized neural processing unit optimized for AI inference. Acrab reports that the Agent Box can achieve prefill rates significantly faster than conventional consumer hardware, demonstrating its capability for complex AI tasks. The company also offers a comprehensive software platform, enabling autonomous agent functionalities and a range of applications beyond the hardware itself.
Acrab's Agent Box aims to decentralize AI by keeping inference on the device. This approach enhances data security, ensures operational continuity during internet outages, and personalizes AI assistants through persistent memory. The company also intends to expand its technology into various products like AI NAS systems, robots, and smart vehicles through strategic partnerships, potentially with entities like CATL, the world's largest EV battery manufacturer.
