triton-inference-server/server
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
3 releases in the last 90 days, the latest 3 days ago, with 143 contributors on GitHub.
Use Cases & Benefits
- Provides an optimized inference serving solution for deploying AI models across cloud, data center, edge, and embedded devices.
- Enables concurrent execution and dynamic batching of models from multiple deep learning and machine learning frameworks for high-performance inferencing.
- Use for deploying real-time AI inference services on NVIDIA GPUs and CPUs in cloud or data center environments.
- Use for building edge AI applications that require efficient inferencing on ARM CPUs or embedded NVIDIA devices like Jetson.
- Use for integrating custom AI model pipelines with business logic scripting and ensemble models to optimize complex workflows.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
BSD 3-Clause
Permissive: commercial use, modification and redistribution are allowed with attribution.
Community files
- README: present
- Contributing guide: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
Triton Inference Server is an open source inference serving software that streamlines AI inferencing. Triton enables teams to deploy any AI model from multiple deep learning and machine learning frameworks, including TensorRT, PyTorch, ONNX, OpenVINO, Python, RAPIDS FIL, and more. Triton Inference Server supports inference across cloud, data center, edge and embedded devices on NVIDIA GPUs, x86 and ARM CPU, or AWS Inferentia. Triton Inference Server delivers optimized performance for many query types, including real time, batched, ensembles and audio/video streaming. Triton inference Server is part of NVIDIA AI Enterprise, a software platform that accelerates the data science pipeline and streamlines the development and deployment of production AI.
New to Triton Inference Server? Make use of these tutorials to begin your Triton journey!
Join the Triton and TensorRT community and stay current on the latest product updates, bug fixes, content, best practices, and more. Need enterprise support? NVIDIA global support is available for Triton Inference Server with the NVIDIA AI Enterprise software suite.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
1.9K+46
Open issues
910+2
Compared with peers in Cloud
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 8, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| Asymptote-Labs/agent-beacon | 3.4K | +315 | 75 | 1 day ago |
| firecracker-microvm/firecracker | 37.9K | — | 62 | 59 days ago |
| vectordotdev/vector | 22.8K | — | 64 | 11 days ago |
| jhipster/generator-jhipster | 22.5K | — | 62 | 51 days ago |
| triton-inference-server/server | 11.1K | — | 60 | 3 days ago |
| moby/buildkit | 10.4K | — | 65 | 1 day ago |
| Category median | 22.5K | — | 64 | — |
Discover Repositories
Search across tracked repositories by name or description