jundot/omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
5 releases in the last 90 days, the latest 1 day ago, with 331 contributors on GitHub.
The biggest single-day gain in the last year, +443 stars on Aug 19, coincided with release v0.6.3rc1.
Use Cases & Benefits
- Provides a local LLM inference server optimized for Apple Silicon with continuous batching and tiered KV caching managed via a macOS menu bar app.
- Enables efficient multi-model serving with hot in-memory and cold SSD caching, preserving context across requests and server restarts for practical local LLM use.
- Use for running and managing multiple LLMs, vision-language, embedding, and reranker models on Apple Silicon with automatic memory and model lifecycle control.
- Use for developers needing a native macOS menubar interface to start, monitor, and auto-update an LLM server without terminal dependency.
- Use for integrating OpenAI-compatible APIs locally with support for tool calling, structured output, and multi-language admin dashboard for real-time model management.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
- Contributing guide: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
LLM inference, optimized for your Mac Continuous batching and tiered KV caching, managed directly from your menu bar.
Every LLM server I tried made me choose between convenience and control. I wanted to pin everyday models in memory, auto-swap heavier ones on demand, set context limits - and manage it all from a menu bar.
oMLX persists KV cache across a hot in-memory tier and cold SSD tier - even when context changes mid-conversation, all past context stays cached and reusable across requests, making local LLMs practical for real coding work with tools like Claude Code. That's why I built it.
Download the
.dmgfrom Releases, drag to Applications, done. The app includes in-app auto-update, so future upgrades are just one click. Note that the macOS app does not install theomlxCLI command. For terminal usage, install via Homebrew or from source.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
+4,645 stars over 82 days across 33 snapshot days; best day +443 on Aug 19.
Forks
2.1K+477
Open issues
579-252
Compared with peers in LLM
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 8, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| NVIDIA/auto-ontology | 11.4K | +8.5K | 75 | — |
| thedotmack/claude-mem | 115.8K | +4.7K | 75 | today |
| DietrichGebert/ponytail | 188.4K | +4.4K | 75 | 12 days ago |
| affaan-m/ECC | 300.8K | +4.3K | 75 | 40 days ago |
| Graphify-Labs/graphify | 145.7K | +3.8K | 75 | today |
| jundot/omlx | 23.3K | — | 60 | 1 day ago |
| Category median | 145.7K | +4.4K | 75 | — |
Discover Repositories
Search across tracked repositories by name or description