vllm-project/vllm-omni
A framework for efficient model inference with omni-modality models
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
Ranks #405 of 2,845 repositories GStars tracks this week (top 14%).
5 releases in the last 90 days, the latest 8 days ago, with 446 contributors on GitHub.
Peaked at #20 in the weekly ranking the week of Jan 4, its best in the past year.
Use Cases & Benefits
- Provides a framework for efficient inference and serving of omni-modality models including text, image, video, and audio.
- Delivers high throughput and low latency by leveraging advanced KV cache management and pipelined stage execution for multimodal model workflows.
- Use for deploying large-scale multimodal AI services that require fast and scalable inference across heterogeneous data types.
- Use for integrating and serving popular Hugging Face omni-modality models with support for distributed inference and streaming outputs.
- Use for research and development of non-autoregressive and diffusion transformer models requiring flexible pipeline abstraction and dynamic resource allocation.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
- Contributing guide: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
Easy, fast, and cheap omni-modality model serving for everyone
Visit our documentation to learn more.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
1.9K+620
Open issues
2.0K+820
Compared with peers in Machine Learning
Fastest-growing repositories GStars tracks in this cohort for the week of Oct 11, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| pytorch/pytorch | 105.0K | +449 | 64 | 23 days ago |
| openbq-org/OpenBB | 74.2K | +0 | 75 | 14 days ago |
| apache/airflow | 47.2K | +0 | 75 | 27 days ago |
| nautechsystems/nautilus_trader | 30.6K | +0 | 75 | 7 days ago |
| GeeeekExplorer/nano-vllm | 15.8K | +0 | 75 | — |
| vllm-project/vllm-omni | 7.2K | +0 | 75 | 8 days ago |
| Category median | 47.2K | +0 | 75 | — |
Also by vllm-project
Other repositories from this maintainer that GStars tracks
- Python95.3K stars
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
- Go6.2K stars
semantic-router
An open, programmable decision layer for models and compute.
- Go5.1K stars
aibrix
Cost-efficient and pluggable Infrastructure components for GenAI inference
- Python3.0K stars
vllm-ascend
Community maintained hardware plugin for vLLM on Huawei Ascend
- JavaScript1.1K stars
recipes
Common recipes to run vLLM
Discover Repositories
Search across tracked repositories by name or description