NVIDIA/Model-Optimizer
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
5 releases in the last 90 days, the latest today, with 124 contributors on GitHub.
Peaked at #3 in the weekly ranking the week of Sep 20, its best in the past year.
Use Cases & Benefits
- Provides a unified library of state-of-the-art model optimization techniques to compress and accelerate deep learning models for inference.
- Enables seamless integration with NVIDIA AI ecosystem and downstream deployment frameworks for optimized inference speed and reduced model size.
- Use for compressing large language models with quantization and pruning to speed up inference on NVIDIA GPUs.
- Use for optimizing diffusion and vision-language models to reduce latency and deployment costs in generative AI applications.
- Use for applying speculative decoding and distillation to improve inference throughput and maintain accuracy in transformer-based models.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
- Contributing guide: present
- Code of conduct: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
The NVIDIA TensorRT Model Optimizer (referred to as Model Optimizer, or ModelOpt) is a library comprising state-of-the-art model optimization techniques including quantization, distillation, pruning, speculative decoding and sparsity to accelerate models.
[Input] Model Optimizer currently supports inputs of a Hugging Face, PyTorch or ONNX model.
[Optimize] Model Optimizer provides Python APIs for users to easily compose the above model optimization techniques and export an optimized quantized checkpoint. Model Optimizer is also integrated with NVIDIA NeMo, Megatron-LM and Hugging Face Accelerate for training required inference optimization techniques.
[Export for deployment] Seamlessly integrated within the NVIDIA AI software ecosystem, the quantized checkpoint generated from Model Optimizer is ready for deployment in downstream inference frameworks like SGLang, TensorRT-LLM, TensorRT, or vLLM.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
+2,096 stars over 82 days across 33 snapshot days; best day +384 on Sep 25.
Forks
800+254
Open issues
463+120
Compared with peers in LLM
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| thedotmack/claude-mem | 113.6K | +8.8K | 35 | today |
| NVIDIA/auto-ontology | 7.5K | +5.6K | 37 | — |
| HKUDS/DeepTutor | 55.5K | +5.5K | 65 | 5 days ago |
| affaan-m/ECC | 297.9K | +5.2K | 63 | 35 days ago |
| DietrichGebert/ponytail | 185.3K | +5.1K | 60 | 7 days ago |
| NVIDIA/Model-Optimizer | 5.5K | — | 45 | today |
| Category median | 113.6K | +5.5K | 60 | — |
Also by NVIDIA
Other repositories from this maintainer that GStars tracks
- Python24.5K stars
SkillSpector
Security scanner for AI agent skills. Detect vulnerabilities, malicious patterns, security risks, prompt injection, data exfiltration, and supply-chain risks in Claude Code, Codex, and MCP skills before you install them.
- TypeScript22.8K stars
NemoClaw
Run agents like Hermes, LangChain Deep Agents, and OpenClaw more securely inside NVIDIA OpenShell with managed inference
- Rust21.2K stars
OpenShell
OpenShell is the safe, private runtime for autonomous AI agents.
- Python18.3K stars
Megatron-LM
Ongoing research training transformer models at scale
- Python15.0K stars
TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
Discover Repositories
Search across tracked repositories by name or description