pytorch/ao
PyTorch native quantization for training and inference
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
1 release in the last 90 days, the latest 88 days ago, with 249 contributors on GitHub.
Gained 18% fewer stars in the last 90 days than in the 90 days before.
Use Cases & Benefits
- Provides PyTorch-native model optimization using quantization and sparsity for efficient training and inference workflows.
- Enables significant speedups and memory reductions with minimal accuracy loss through advanced quantization-aware training and float8 support.
- Use for accelerating large-scale LLM training with float8 precision to reduce GPU memory and improve throughput.
- Use for deploying quantized transformer models with int4 weights to achieve faster inference and lower memory usage on GPUs.
- Use for integrating sparsity techniques like 2:4 and block sparsity to speed up vision transformer training with minimal accuracy drop.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Other
GitHub could not identify this license; read the LICENSE file before depending on it.
Community files
- README: present
- Contributing guide: present
- Code of conduct: present
Watch-outs
- License not identified by GitHub; read the LICENSE file before depending on it.
General guidance from repository metadata, not legal advice. How we score
From the README
TorchAO is a PyTorch-native model optimization framework leveraging quantization and sparsity to provide an end-to-end, training-to-serving workflow for AI models. TorchAO works out-of-the-box with
torch.compile()andFSDP2across most HuggingFace PyTorch models. Key features include:Check out our docs for more details!
First, install TorchAO. We recommend installing the latest stable version:
Quantize your model weights to int4!
Compared to a
torch.compiledbf16 baseline, your quantized model should be significantly smaller and faster on a single A100 GPU:For the full model setup and benchmark details, check out our quick start guide. Alternatively, try quantizing your favorite model using our HuggingFace space!
Please see the torchao compability table for version requirements for dependencies.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
666+87
Open issues
828-27
Compared with peers in LLM
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| thedotmack/claude-mem | 113.6K | +8.8K | 35 | today |
| NVIDIA/auto-ontology | 7.5K | +5.6K | 37 | — |
| HKUDS/DeepTutor | 55.5K | +5.5K | 65 | 5 days ago |
| affaan-m/ECC | 297.9K | +5.2K | 63 | 35 days ago |
| DietrichGebert/ponytail | 185.3K | +5.1K | 60 | 7 days ago |
| pytorch/ao | 3.0K | — | 60 | 88 days ago |
| Category median | 113.6K | +5.5K | 60 | — |
Also by pytorch
Other repositories from this maintainer that GStars tracks
Discover Repositories
Search across tracked repositories by name or description