optillm
OptiLLM is a OpenAI API compatible inference and optimization server that does not require model training and increases the success rate in reasoning tasks. The software supports 20 more optimization techniques, such as best-of-N, MCTS, planning and Mixture of Agents. Thanks to LiteLLM integration, OpenAI, Anthropic, Google and can run over 100 models via proxy.
algorithmicsuperintelligence · API Gateways & Clients
Common commercial products in this category: Apigee, AWS API Gateway, Azure API Management
- By routing existing API calls through OptiLLM, you increase accuracy in math, coding, and logical reasoning tasks without model training.
- In OpenAI compatible clients, you add a prefix to the model names, enabling advanced optimization techniques such as Mixture of Agents or similar.
- With LiteLLM support, you manage over 100 different models, including OpenAI, Anthropic and Google, via a single proxy endpoint.
- • NeutralHacker News
Optillm: An Optimizing Inference Proxy with Plugins
“Optillm is an optimizing inference proxy that has over a dozen techniques that aim to improve the accuracy of the responses using test-time compute. Over the last couple of months we have set several SOTA results using smaller and less capable models like gpt-4o-mini. Recently,…”
- • NeutralGitHub Issue
Can't install z3-solver, is it possible to support lean4?
“Hello 1.Can't install z3-solver, is it possible to support lean4? 2.Is it possible to support alphageometry? For geometry problems, route to alphageometry, but I don't know how much effort to use it?”
My note
Alternative pages
More in: API Gateways & Clients
- Caddy★ 78,586 · yesterday
- Kong Gateway★ 44,332 · 12 days ago
- HTTPie★ 38,687 · 1 yr ago
- Nginx Proxy Manager★ 34,592 · 10 days ago
- Nginx★ 32,017 · yesterday
- Envoy Proxy★ 29,159 · today