vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
Ranks #50 of 2,841 repositories GStars tracks this week (top 2%).
5 releases in the last 90 days, the latest 2 days ago, with 453 contributors on GitHub.
Peaked at #30 in the weekly ranking the week of Aug 31, its best in the past year.
Use Cases & Benefits
- Provides a high-throughput, memory-efficient inference and serving engine for large language models (LLMs).
- Delivers state-of-the-art serving speed and memory optimization using techniques like PagedAttention and CUDA/HIP graph execution.
- Use for deploying and serving transformer-based LLMs with efficient resource utilization in production environments.
- Use for integrating multi-modal and mixture-of-expert LLMs with scalable distributed inference across GPUs and TPUs.
- Use for building OpenAI-compatible API servers that support streaming outputs and various decoding algorithms.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
- Contributing guide: present
- Code of conduct: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
Easy, fast, and cheap LLM serving for everyone
vLLM is a fast and easy-to-use library for LLM inference and serving.
Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has evolved into a community-driven project with contributions from both academia and industry.
vLLM seamlessly supports most popular open-source models on HuggingFace, including:
Find the full list of supported models here.
Install vLLM with
pipor from source:Visit our documentation to learn more.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
+7,019 stars over 85 days across 12 snapshot days; best day +40 on Aug 14.
Forks
23.1K+3.7K
Open issues
8.6K+2.8K
Compared with peers in AI
Fastest-growing repositories GStars tracks in this cohort for the week of Oct 4, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| ifixai-ai/iFixAi | 23.1K | +4.2K | 56 | 2 days ago |
| heygen-com/hyperframes | 62.8K | +4.0K | 45 | today |
| calesthio/OpenMontage | 68.0K | +3.8K | 75 | — |
| thedotmack/claude-mem | 100.4K | +3.7K | 75 | today |
| rohitg00/ai-engineering-from-scratch | 68.7K | +3.5K | 75 | 3 days ago |
| vllm-project/vllm | 93.4K | +0 | 75 | 2 days ago |
| Category median | 68.0K | +3.8K | 75 | — |
Also by vllm-project
Other repositories from this maintainer that GStars tracks
- Python7.1K stars
vllm-omni
A framework for efficient model inference with omni-modality models
- Go6.1K stars
semantic-router
An open, programmable decision layer for models and compute.
- Go5.1K stars
aibrix
Cost-efficient and pluggable Infrastructure components for GenAI inference
- Python2.9K stars
vllm-ascend
Community maintained hardware plugin for vLLM on Huawei Ascend
- JavaScript1.0K stars
recipes
Common recipes to run vLLM
Discover Repositories
Search across tracked repositories by name or description