LiteLLM
Unified OpenAI-compatible proxy for 100+ LLM providers with cost tracking and load balancing
LiteLLM is a Python SDK and proxy server that provides a single OpenAI-compatible interface to call over 100 LLM APIs, including OpenAI, Anthropic, Azure, Bedrock, Vertex AI, Cohere, and Ollama. It handles cost tracking, budget management, virtual API keys, guardrails, and load balancing across deployments. The proxy can handle 1,500+ requests per second. The core SDK and proxy are open source under MIT, with enterprise features available for teams needing SSO and advanced auth.
Pricing: Free / enterprise
LiteLLM is a gateway that puts one OpenAI-compatible API in front of 100+ model providers, so switching between them is a configuration change rather than a code change. Beyond translation it handles the operational layer: virtual keys, per-key and per-team budgets, spend tracking, rate limits, fallbacks and logging. Endpoints cover chat completions, messages, embeddings, images, audio, batches and rerank.
The licensing needs care. The main tree is MIT, but the enterprise/ directory carries a separate BerriAI commercial licence that permits copying and modification for development and testing only, and requires a paid seat-based subscription for production use. The features behind that line are the ones an enterprise typically assumes are included: SSO and SCIM, OIDC and JWT auth, audit logs, secret managers with key rotation, organisation and team admin controls, multi-region control plane and 24/7 support. Enterprise pricing is quoted annually against gateway request capacity and deployment architecture rather than per token, with no public figure. Audit whether a deployment touches enterprise/ before going to production.
The project moves quickly and its breadth shows: roughly 4,900 issues are open against 56,000 stars, so provider-specific edge cases are common. That is the trade for covering this many providers behind one interface.
LiteLLM Alternatives
Explore 37 products in the Frameworks & Stacks category. View all LiteLLM alternatives.
llama.cpp
LLM inference in C/C++ with broad hardware support and aggressive quantization
Mastra
TypeScript-first AI framework for building agents, RAG pipelines, and workflows
Ollama
Run large language models locally with a single command
CC Switch
Open-source desktop manager and local router for AI coding tools
Work on LiteLLM? Feature it at the top of Frameworks & Stacks.
Is your product missing?