ai-dynamo/dynamo
A Datacenter Scale Distributed Inference Serving Framework
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
5 releases in the last 90 days, the latest 2 days ago, with 427 contributors on GitHub.
Peaked at #42 in the weekly ranking the week of Jan 11, its best in the past year.
Use Cases & Benefits
- Provides a high-throughput, low-latency distributed inference serving framework for generative AI and reasoning models across multi-node GPU clusters.
- Enables efficient orchestration of large language model shards with dynamic GPU scheduling and KV cache-aware request routing to maximize throughput and minimize latency.
- Use for deploying scalable multi-GPU, multi-node LLM inference services with disaggregated prefill and decode capabilities on NVIDIA hardware.
- Use for integrating diverse inference engines like vLLM, SGLang, and TensorRT-LLM in a unified, engine-agnostic serving platform with optimized data transfer.
- Use for building OpenAI-compatible HTTP API servers that route and load balance requests to multiple LLM workers in data center environments.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Other
GitHub could not identify this license; read the LICENSE file before depending on it.
Community files
- README: present
- Contributing guide: present
- Code of conduct: present
Watch-outs
- License not identified by GitHub; read the LICENSE file before depending on it.
General guidance from repository metadata, not legal advice. How we score
From the README
High-throughput, low-latency inference framework designed for serving generative AI and reasoning models in multi-node distributed environments.
Large language models are quickly outgrowing the memory and compute budget of any single GPU. Tensor-parallelism solves the capacity problem by spreading each layer across many GPUs—and sometimes many servers—but it creates a new one: how do you coordinate those shards, route requests, and share KV cache fast enough to feel like one accelerator? This orchestration gap is exactly what NVIDIA Dynamo is built to close.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
+700 stars over 86 days across 38 snapshot days; best day +22 on Sep 14.
Forks
1.8K+391
Open issues
2.4K+1.4K
Compared with peers in Backend
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| anthropics/skills | 187.8K | +2.9K | 15 | — |
| DuarteSantos8/openGym | 19.7K | +2.4K | 60 | 2 days ago |
| shiyu-coder/Kronos | 47.0K | +2.3K | 16 | — |
| NationalSecurityAgency/ghidra | 98.2K | +2.3K | 64 | 14 days ago |
| microsoft/VibeVoice | 56.0K | +1.1K | 29 | — |
| ai-dynamo/dynamo | 8.4K | — | 65 | 2 days ago |
| Category median | 56.0K | +2.3K | 29 | — |
Also by ai-dynamo
Other repositories from this maintainer that GStars tracks
Discover Repositories
Search across tracked repositories by name or description