vllm-project/vllm-ascend
Community maintained hardware plugin for vLLM on Huawei Ascend
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
Ranks #318 of 3,103 repositories GStars tracks this week (top 10%).
5 releases in the last 90 days, the latest 6 days ago, with 448 contributors on GitHub.
Gained 13% more stars in the last 90 days than in the 90 days before.
Use Cases & Benefits
- Provides a hardware plugin enabling seamless integration of vLLM with Huawei Ascend NPUs for efficient large language model inference.
- Offers a hardware-pluggable interface that decouples Ascend NPU integration, supporting diverse open-source models including Transformers and Mixture-of-Experts.
- Use for deploying large-scale expert parallelism models on Ascend NPUs to accelerate inference workloads in production environments.
- Use for fine-tuning and evaluating multi-modal and embedding-based LLMs on Ascend hardware within MLOps pipelines.
- Use for integrating Ascend NPU support into existing vLLM workflows to leverage specialized hardware for transformer-based model serving.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
- Contributing guide: present
- Code of conduct: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
vLLM Ascend Plugin
vLLM Ascend (
vllm-ascend) is a community maintained hardware plugin for running vLLM seamlessly on the Ascend NPU.It is the recommended approach for supporting the Ascend backend within the vLLM community. It adheres to the principles outlined in the [RFC]: Hardware pluggable, providing a hardware-pluggable interface that decouples the integration of the Ascend NPU with vLLM.
By using vLLM Ascend plugin, popular open-source models, including Transformer-like, Mixture-of-Expert, Embedding, Multi-modal LLMs can run seamlessly on the Ascend NPU.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
2.5K+702
Open issues
4.2K+1.6K
Compared with peers in LLM
Fastest-growing repositories GStars tracks in this cohort for the week of Oct 18, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| DietrichGebert/ponytail | 174.8K | +9.9K | 75 | 7 days ago |
| xerj-org/xerj | 14.2K | +7.8K | 75 | today |
| affaan-m/ECC | 287.3K | +6.4K | 64 | 22 days ago |
| tashfeenahmed/freellmapi | 40.3K | +4.1K | 75 | today |
| JuliusBrussee/caveman | 114.4K | +2.2K | 75 | 9 days ago |
| vllm-project/vllm-ascend | 3.0K | +0 | 75 | 6 days ago |
| Category median | 114.4K | +6.4K | 75 | — |
Also by vllm-project
Other repositories from this maintainer that GStars tracks
- Python95.3K stars
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
- Python7.2K stars
vllm-omni
A framework for efficient model inference with omni-modality models
- Go6.2K stars
semantic-router
An open, programmable decision layer for models and compute.
- Go5.1K stars
aibrix
Cost-efficient and pluggable Infrastructure components for GenAI inference
- JavaScript1.1K stars
recipes
Common recipes to run vLLM
Discover Repositories
Search across tracked repositories by name or description