NVIDIA/Megatron-LM
Ongoing research training transformer models at scale
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
Ranks #22 among LLM repositories tracked this week, and #22 over the last 30 days; #482 of 2,794 repositories GStars tracks this week (top 17%).
5 releases in the last 90 days, the latest today, with 342 contributors on GitHub.
The biggest single-day gain in the last year, +112 stars on Feb 26, coincided with release core_v0.16.0.
Use Cases & Benefits
- Provides a GPU-optimized library and reference implementation for training large-scale transformer models with advanced parallelism and mixed precision support.
- Enables efficient training of multi-billion parameter models at scale with modular, composable components and state-of-the-art performance optimizations on NVIDIA GPUs.
- Use for training state-of-the-art foundation language models like GPT, LLaMA, and MoE models across thousands of GPUs with high throughput and fault tolerance.
- Use for developing custom distributed training frameworks requiring flexible parallelism strategies such as tensor, pipeline, data, context, and expert parallelism.
- Use for research teams experimenting with new transformer architectures, training techniques, and mixed precision formats including FP8 on latest NVIDIA hardware.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Other
GitHub could not identify this license; read the LICENSE file before depending on it.
Community files
- README: present
- Contributing guide: present
Watch-outs
- License not identified by GitHub; read the LICENSE file before depending on it.
General guidance from repository metadata, not legal advice. How we score
From the README
GPU-optimized library for training transformer models at scale
→ Complete Installation Guide - Docker, pip variants (dev,lts,etc.), source installation, and system requirements
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
+1,048 stars over 90 days across 12 snapshot days; best day +16 on Aug 28.
Forks
4.6K+381
Open issues
1.6K+628
Compared with peers in LLM
Fastest-growing repositories GStars tracks in this cohort for the week of Oct 18, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| DietrichGebert/ponytail | 173.5K | +9.2K | 75 | 6 days ago |
| xerj-org/xerj | 13.1K | +8.1K | 75 | today |
| tashfeenahmed/freellmapi | 39.6K | +3.9K | 75 | 1 day ago |
| JuliusBrussee/caveman | 114.4K | +2.2K | 75 | 9 days ago |
| thesysdev/openui | 13.1K | +2.1K | 75 | 7 days ago |
| NVIDIA/Megatron-LM | 18.1K | +0 | 40 | today |
| Category median | 39.6K | +3.9K | 75 | — |
Also by NVIDIA
Other repositories from this maintainer that GStars tracks
- TypeScript22.7K stars
NemoClaw
Run agents like Hermes, LangChain Deep Agents, and OpenClaw more securely inside NVIDIA OpenShell with managed inference
- Python20.5K stars
SkillSpector
Security scanner for AI agent skills. Detect vulnerabilities, malicious patterns, security risks, prompt injection, data exfiltration, and supply-chain risks in Claude Code, Codex, and MCP skills before you install them.
- Rust17.8K stars
OpenShell
OpenShell is the safe, private runtime for autonomous AI agents.
- Python14.9K stars
TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
- 12.1K stars
cosmos
NVIDIA Cosmos is an open platform of world models, datasets, and tools that enables developers to build Physical AI for robots, autonomous vehicles, smart infrastructure, and more.
Discover Repositories
Search across tracked repositories by name or description