AI Cloud Startup Secures $400M Loan for Inference Chips

General Compute, a startup focused on AI inference cloud services, has secured a substantial $400 million loan from Upper90, a prominent tech investment firm. This financial agreement is distinguished by its potential to be the first of its kind, utilizing inference-specific chips as collateral. These chips are designed for the efficient execution of pre-trained AI models, differentiating them from the more costly chips used for AI model development.

The financing reflects a growing market concern regarding the high cost of AI tools. Investors are increasingly favoring infrastructure that supports open-source models at a lower price point compared to the latest large language models from leading labs. General Compute, founded by CEO Finn Puklowski, previously raised a $15 million seed round in May to establish an inference 'neocloud' utilizing silicon from SambaNova.

These specialized SN50 chips are engineered for efficiency, requiring less power and avoiding expensive water-cooling systems. General Compute asserts these chips will deliver inference speeds up to 16 times faster than GPU-based clouds. The company's ability to procure these chips is crucial, especially for a new entrant.

Upper90's CEO, Billy Libby, has prior experience in similar financing, having supported Crusoe with loans against GPU purchases in 2021. While traditional lenders were hesitant, the success of this model, exemplified by CoreWeave, has normalized chip-backed financing. Libby views this as an opportunity to invest in the next wave of AI, focusing on inference and open-source models.

This strategic focus aligns with market trends, as companies offering access to open models like OpenRouter and Fireworks have recently secured significant funding. The emergence of competitive open-source models and alternative chipmakers like Groq and Cerebras further validates the market's demand for diverse and cost-effective AI solutions outside of Nvidia's ecosystem.

Puklowski highlighted that this deal signifies a broader capital movement towards fragmenting Nvidia's market dominance, enabling the adoption of cost-efficient inference solutions by leveraging emerging chip technologies with superior total cost of ownership.