flashinfer-ai/flashinfer
FlashInfer: Kernel Library for LLM Serving
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
5 releases in the last 90 days, the latest 1 day ago, with 432 contributors on GitHub.
Use Cases & Benefits
- Provides a high-performance CUDA kernel library for efficient Large Language Model (LLM) serving and inference.
- Delivers state-of-the-art GPU kernels with customizable attention variants and memory-efficient scheduling for low-latency LLM inference.
- Use for accelerating LLM inference workloads requiring sparse and dense attention on NVIDIA GPUs with PyTorch integration.
- Use for developing custom attention mechanisms via JIT compilation to optimize novel LLM architectures and research.
- Use for integrating efficient LLM serving kernels into distributed inference systems needing load-balanced scheduling and low-precision support.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
- Contributing guide: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
FlashInfer is a library and kernel generator for Large Language Models that provides high-performance implementation of LLM GPU kernels such as FlashAttention, SparseAttention, PageAttention, Sampling, and more. FlashInfer focuses on LLM serving and inference, and delivers state-of-the-art performance across diverse scenarios.
Check our v0.2 release blog for new features!
FlashInfer supports PyTorch, TVM and C++ (header-only) APIs, and can be easily integrated into existing projects.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
1.7K+451
Open issues
1.4K+466
Compared with peers in Graphics
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| heygen-com/hyperframes | 79.6K | +4.1K | 45 | today |
| mrdoob/three.js | 117.7K | +463 | 65 | 6 days ago |
| NVIDIA/cuda-rust | 4.0K | +122 | 60 | 4 days ago |
| bilawalsidhu/gods-eye-view | 61.8K | — | 60 | 3 days ago |
| processing/p5.js | 24.2K | — | 65 | 7 days ago |
| flashinfer-ai/flashinfer | 6.8K | — | 60 | 1 day ago |
| Category median | 61.8K | +463 | 60 | — |
Discover Repositories
Search across tracked repositories by name or description