kubernetes-sigs/gateway-api-inference-extension
Gateway API Inference Extension
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
3 releases in the last 90 days, the latest 49 days ago, with 170 contributors on GitHub.
Use Cases & Benefits
- Extends Envoy-capable gateways to become inference gateways for optimized routing and load balancing of self-hosted generative AI models on Kubernetes.
- Enables Kubernetes-native declarative APIs for sophisticated routing, traffic splitting, and operational guardrails to improve AI inference workload management and utilization.
- Use for deploying and managing large language model inference workloads with improved latency and throughput on Kubernetes clusters.
- Use for integrating self-hosted AI models alongside model-as-a-service providers in hybrid AI gateway environments.
- Use for implementing cost- and performance-aware scheduling of inference requests across heterogeneous accelerators and model servers.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
- Contributing guide: present
- Code of conduct: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
Gateway API Inference Extension optimizes self-hosting Generative Models on Kubernetes. This is achieved by leveraging Envoy's External Processing (ext-proc) to extend any gateway that supports both ext-proc and Gateway API into an inference gateway.
NOTE : As we prep for our
v1release, some of our docs may fall out of scope, we are working hard to get these up to date and they will be ready by the time we launchv1. Thanks!Inference Gateway has partnered with vLLM to accelerate LLM serving optimizations with llm-d!
The following specific terms to this project:
The following are key industry terms that are important to understand for this project:
For deeper insights and more advanced concepts, refer to our proposals.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
324+19
Open issues
26+4
Compared with peers in API
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| vxcontrol/pentagi | 33.5K | +712 | 38 | 31 days ago |
| usebruno/bruno | 48.2K | +281 | 59 | 14 days ago |
| agentgateway/agentgateway | 6.0K | +183 | 60 | 9 days ago |
| mudler/LocalAI | 50.2K | +176 | 59 | 12 days ago |
| sveltejs/svelte | 88.6K | +77 | 65 | 9 days ago |
| kubernetes-sigs/gateway-api-inference-extension | 801 | — | 62 | 49 days ago |
| Category median | 48.2K | +183 | 59 | — |
Also by kubernetes-sigs
Other repositories from this maintainer that GStars tracks
- Go15.6K stars
kind
Kubernetes IN Docker - local clusters for testing Kubernetes
- Go12.2K stars
kustomize
Customization of kubernetes YAML configurations
- Go9.4K stars
kubebuilder
Kubebuilder - SDK for building Kubernetes APIs using CRDs
- Go9.1K stars
external-dns
Configure external DNS servers dynamically from Kubernetes resources
- TypeScript7.6K stars
headlamp
A Kubernetes web UI that is fully-featured, user-friendly and extensible
Discover Repositories
Search across tracked repositories by name or description