FareedKhan-dev/train-llm-from-scratch
A straightforward method for training your LLM, from downloading data to generating text.
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
Ranks #20 among LLM repositories tracked this week, and #20 over the last 30 days; #462 of 2,845 repositories GStars tracks this week (top 16%).
The project lists 13 contributors on GitHub.
Peaked at #13 in the weekly ranking the week of Sep 27, its best in the past year.
Use Cases & Benefits
- Implements a complete pipeline to train large language models from raw text to aligned reasoning models using plain PyTorch without external transformer libraries.
- Enables training billion-parameter LLMs on a single GPU with fully transparent, hand-written algorithms for every step from tokenization to reinforcement learning.
- Use for educational purposes to learn transformer internals and LLM training by studying and running clean, modular PyTorch code.
- Use for researchers experimenting with custom reward models and reinforcement learning algorithms like PPO, DPO, and GRPO on small to mid-size transformers.
- Use for developers needing a lightweight, end-to-end LLM training framework that supports supervised fine-tuning and evaluation without heavy dependencies.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
MIT
Permissive: commercial use, modification and redistribution are allowed with attribution.
Community files
- README: present
- Contributing guide: present
Watch-outs
- No GitHub releases; pin a commit or tag instead of a release.
General guidance from repository metadata, not legal advice. How we score
From the README
I am Looking for a PhD position in AI. GitHub
I implemented a transformer model from scratch using PyTorch, based on the paper Attention is All You Need. You can use my scripts to train your own billion or million parameter LLM using a single GPU.
This started as a pretraining tutorial. It now goes all the way from raw text to an aligned, reasoning style model, with every algorithm hand written in plain PyTorch (no
trl, nopeft, notransformers). The whole journey is one idea repeated: turn text into numbers, predict the next token, then keep changing the data and the loss until the model does what we want.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
+1,936 stars over 26 days across 10 snapshot days; best day +321 on Sep 22.
Forks
1.7K+261
Open issues
1-6
Compared with peers in LLM
Fastest-growing repositories GStars tracks in this cohort for the week of Oct 11, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| DietrichGebert/ponytail | 173.5K | +5.3K | 75 | 6 days ago |
| tashfeenahmed/freellmapi | 39.6K | +3.8K | 75 | 1 day ago |
| thedotmack/claude-mem | 102.3K | +2.4K | 75 | 2 days ago |
| TencentCloud/TencentDB-Agent-Memory | 29.7K | +1.5K | 75 | 7 days ago |
| p-e-w/heretic | 35.5K | +1.3K | 41 | 124 days ago |
| FareedKhan-dev/train-llm-from-scratch | 12.1K | +0 | 75 | — |
| Category median | 39.6K | +2.4K | 75 | — |
Discover Repositories
Search across tracked repositories by name or description