Home / Technology / OpenAI Enters Fast Decision API Market
OpenAI Enters Fast Decision API Market
23 Oct
Summary
- OpenAI launched its Decisions API for faster, cheaper AI model inferences.
- The API aims to replace slower, more expensive large language models.
- Competitors like TypeSafe offer specialized decision models with high benchmarks.
OpenAI has made its Decisions API widely available, marking its entry into the rapidly growing market for fast and economical AI decision-making. This new API allows software developers to replace existing large language models (LLMs) in various workflows, offering dramatically reduced costs and improved speed. The Decisions API achieved this by reconfiguring models like ChatGPT Luna to enhance inference speed and lower costs.
The key advantage of OpenAI's Decisions API is its speed, shrinking response times from one to three seconds down to approximately 100-180 milliseconds. This improvement has already been demonstrated by early customers, with some seeing their time between actions cut from 1.4 seconds to just 0.13 seconds. OpenAI itself has used the product to enhance the responsiveness of its Dots voice assistant.
However, the market for decision models is competitive, with startups like TypeSafe offering specialized models. TypeSafe's Jev model has gained traction, and benchmarks show other models from Liquid and Base Lab performing strongly. While OpenAI's Decisions API shows promise, particularly in multi-modality, the optimal decision model can vary by workload, with users encouraged to trial different options for their specific needs.
