BerriAI/litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
5 releases in the last 90 days, the latest 4 days ago, with 370 contributors on GitHub.
The biggest single-day gain in the last year, +334 stars on Oct 10, coincided with release v1.106.0-dev.3.
Use Cases & Benefits
- Provides a lightweight AI gateway to call 100+ large language model APIs using a unified OpenAI-compatible interface with Rust core and Python SDK.
- Enables cost tracking, guardrails, load balancing, and consistent output formatting across multiple LLM providers in a single proxy server.
- Use for integrating multiple LLM providers like OpenAI, Anthropic, Azure, and Bedrock into applications with unified API calls and fallback logic.
- Use for managing API usage budgets, rate limits, and logging observability across teams and projects via a self-hosted or hosted proxy server.
- Use for streaming and asynchronous LLM completions in Python applications requiring scalable, multi-provider AI model access with detailed usage tracking.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Other
GitHub could not identify this license; read the LICENSE file before depending on it.
Community files
- README: present
- Contributing guide: present
Watch-outs
- License not identified by GitHub; read the LICENSE file before depending on it.
General guidance from repository metadata, not legal advice. How we score
From the README
Call all LLM APIs using the OpenAI format [Bedrock, Huggingface, VertexAI, TogetherAI, Azure, OpenAI, Groq etc.]
🚨 Stable Release: Use docker images with the
-stabletag. These have undergone 12 hour load tests, before being published. More information about the release cycle hereSupport for more providers. Missing a provider or LLM Platform, raise a feature request.
Call any model supported by a provider, with
model=<provider_name>/<model_name>. There might be provider-specific details here, so refer to provider docs for more information
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
+6,968 stars over 90 days across 19 snapshot days; best day +334 on Oct 10.
Forks
13.0K+2.6K
Open issues
5.7K+886
Compared with peers in LLM
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 8, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| NVIDIA/auto-ontology | 11.4K | +8.5K | 75 | — |
| thedotmack/claude-mem | 115.8K | +4.7K | 75 | today |
| DietrichGebert/ponytail | 188.4K | +4.4K | 75 | 12 days ago |
| affaan-m/ECC | 300.8K | +4.3K | 75 | 40 days ago |
| Graphify-Labs/graphify | 145.7K | +3.8K | 75 | today |
| BerriAI/litellm | 62.6K | — | 60 | 4 days ago |
| Category median | 145.7K | +4.4K | 75 | — |
Discover Repositories
Search across tracked repositories by name or description