mostlygeek/llama-swap
Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don't need it.
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
5 releases in the last 90 days, the latest today, with 87 contributors on GitHub.
Peaked at #40 in the weekly ranking the week of Mar 29, its best in the past year.
Use Cases & Benefits
- Provides a lightweight proxy server for automatic model swapping across local OpenAI/Anthropic compatible servers like llama.cpp and vllm.
- Enables seamless on-demand switching and concurrent running of multiple models with minimal configuration and no dependencies.
- Use for managing multiple local language models with automatic loading and unloading to optimize resource usage.
- Use for integrating various OpenAI-compatible inference servers behind a unified API with transparent model routing.
- Use for monitoring and controlling model activity via a real-time web UI and remote log streaming.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
MIT
Permissive: commercial use, modification and redistribution are allowed with attribution.
Community files
- README: present
- Contributing guide: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
llama-swap is a light weight, transparent proxy server that provides automatic model swapping to llama.cpp's server.
Written in golang, it is very easy to install (single binary with no dependencies) and configure (single yaml file). To get started, download a pre-built binary, a provided docker images or Homebrew.
When a request is made to an OpenAI compatible endpoint, llama-swap will extract the
modelvalue and load the appropriate server configuration to serve it. If the wrong upstream server is running, it will be replaced with the correct one. This is where the "swap" part comes in. The upstream server is automatically swapped to the correct one to serve the request.In the most basic configuration llama-swap handles one model at a time. For more advanced use cases, the
groupsfeature allows multiple models to be loaded at the same time. You have complete control over how your system resources are used.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
+845 stars over 84 days across 13 snapshot days; best day +13 on Aug 17.
Forks
541+128
Open issues
119+42
Compared with peers in LLM
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| thedotmack/claude-mem | 113.6K | +8.8K | 35 | today |
| NVIDIA/auto-ontology | 7.5K | +5.6K | 37 | — |
| HKUDS/DeepTutor | 55.5K | +5.5K | 65 | 5 days ago |
| affaan-m/ECC | 297.9K | +5.2K | 63 | 35 days ago |
| DietrichGebert/ponytail | 185.3K | +5.1K | 60 | 7 days ago |
| mostlygeek/llama-swap | 6.2K | — | 40 | today |
| Category median | 113.6K | +5.5K | 60 | — |
Discover Repositories
Search across tracked repositories by name or description