lightseekorg/tokenspeed
TokenSpeed is a speed-of-light LLM inference engine.
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
1 release in the last 90 days, the latest 23 days ago, with 105 contributors on GitHub.
Peaked at #34 in the weekly ranking the week of Oct 11, its best in the past year.
Use Cases & Benefits
- Delivers a high-performance LLM inference engine optimized for agentic workloads with speed-of-light execution.
- Offers TensorRT-LLM-level speed combined with vLLM-level usability, enabling efficient and scalable inference without manual parallelism coding.
- Use for deploying large language models in production environments requiring ultra-low latency and high throughput.
- Use for research and development of agentic AI systems that demand fine-grained control over request lifecycle and resource management.
- Use for integrating cutting-edge kernel optimizations and multi-head latent attention on GPU platforms like NVIDIA and AMD for advanced model inference.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
MIT
Permissive: commercial use, modification and redistribution are allowed with attribution.
Community files
- README: present
- Contributing guide: present
- Code of conduct: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
TokenSpeed is a speed-of-light LLM inference engine designed for agentic workloads, with TensorRT-LLM-level performance and vLLM-level usability. Our goal is to be the most performant inference engine for production agentic workloads.
For technical blogs, conference talks, and engineering articles from LightSeek Foundation, visit the LightSeek Blog.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
+981 stars over 80 days across 19 snapshot days; best day +232 on Oct 16.
Forks
387+156
Open issues
114+63
Compared with peers in LLM
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| thedotmack/claude-mem | 113.6K | +8.8K | 35 | today |
| NVIDIA/auto-ontology | 7.5K | +5.6K | 37 | — |
| HKUDS/DeepTutor | 55.5K | +5.5K | 65 | 5 days ago |
| affaan-m/ECC | 297.9K | +5.2K | 63 | 35 days ago |
| DietrichGebert/ponytail | 185.3K | +5.1K | 60 | 7 days ago |
| lightseekorg/tokenspeed | 2.8K | — | 64 | 23 days ago |
| Category median | 113.6K | +5.5K | 60 | — |
Discover Repositories
Search across tracked repositories by name or description