ashhart/TensorFold
LLM Inference Engine for Metal, CUDA and Vulkan.
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
5 releases in the last 90 days, the latest 1 day ago, with 123 contributors on GitHub.
Use Cases & Benefits
- Serves large language models on Apple Silicon and NVIDIA GPUs via an OpenAI-compatible API with exact decoding and draft verification.
- Enables fast, exact LLM inference optimized for Apple Silicon's MLX and CUDA-enabled NVIDIA GPUs with support for multiple quantized checkpoint formats.
- Use for deploying type-safe, high-throughput LLM chat and completion services on macOS with native MLX acceleration.
- Use for running large-scale LLM inference on NVIDIA GPUs with CUDA support, including multi-rank setups and experimental quantized models.
- Use for integrating image input capabilities into LLMs using supported GLM and Qwen models with vision extensions on both MLX and CUDA backends.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
- Contributing guide: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
TensorFold serves language models on Apple Silicon and NVIDIA GPUs through an OpenAI-compatible API. Each model family supplies its own kernels and draft verification.
On a Mac, Homebrew installs it too:
brew install ashhart/tensorfold/tensorfold.Use
http://127.0.0.1:8080/v1as the client base URL and the model ID from/v1/models. Both backends serve chat completions, completions, OpenAI Responses (/v1/responses) and Anthropic Messages (/v1/messages); see the API reference. Python 3.11 or newer is required, and MLX 0.32.2 or newer on a Mac (pip installs it). See the runbook for installation and a first request. On NVIDIA GPUs the CUDA kernels need compute capability 8.9 or newer: Ada (RTX 40 series), Hopper and Blackwell, including the DGX Spark's GB10 and the RTX 50 series. NVFP4 and FP8 checkpoints run from 8.9: their own math where the GPU has each mma (FP4 on 12.x, FP8 from 8.9), W4A16 elsewhere; the RTX 40, Hopper and B200 builds are compiled and bit-checked on Blackwell but not yet run on those cards. RTX 30 cards (8.6) aren't supported, and the server refuses a GPU below 8.9 at startup.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
+1,105 stars over 38 days across 7 snapshot days; best day +36 on Oct 7.
Forks
352+298
Open issues
312+300
Compared with peers in AI
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| thedotmack/claude-mem | 113.6K | +8.8K | 35 | today |
| D4Vinci/Scrapling | 98.1K | +6.9K | 65 | 1 day ago |
| Leonxlnx/taste-skill | 110.7K | +5.2K | 55 | 1 day ago |
| calesthio/OpenMontage | 77.2K | +4.2K | 40 | — |
| heygen-com/hyperframes | 79.6K | +4.1K | 45 | today |
| ashhart/TensorFold | 1.7K | — | 60 | 1 day ago |
| Category median | 98.1K | +5.2K | 45 | — |
Discover Repositories
Search across tracked repositories by name or description