nanoGPT
nanoGPT is a plain PyTorch repository written to train and fine-tuning medium-sized GPT models; it contains about 300 lines of training cycles and 300 lines of model definition, OpenAI can load GPT-2 weights. train.py reproduces the GPT-2 (124M) model on OpenWebText. As of November 2025, the author declared the repository old and deprecated, suggesting a nanochat instead.
nanoGPT
69karpathy · AI / LLM / RAG
Common commercial products in this category: ChatGPT, Claude, Gemini
- Based on Shakespeare data, you quickly train a character-based little GPT model.
- You use GPT-2 weights supplied by OpenAI by uploading them to the model
- You do fine-tuning (finetune) over pre-trained checkpoints
- • NeutralHacker News
karpathy/build-nanogpt: Video + code lecture on building nanoGPT from scratch
“H1: build nanoGPT Title: Video+code lecture on building nanoGPT from scratch Please don't do things to make titles stand out (..) Otherwise please use the original title”
- • NeutralGitHub Issue
Training on M1 "MPS"
“Most of the people do not have access to 8XA100 40GB systems. But a single M1 Max laptop with 64 GB memory could host the training. How difficult is it to port this code to "MPS" ?”
Verified against README · Oct 26, 2026 · source
My note
Alternative pages
More in: AI / LLM / RAG
- rea★ 412,812 · today
- ECC★ 293,487 · yesterday
- hermes-agent★ 258,923 · today
- ponytail★ 181,048 · 2 days ago
- graphify★ 139,557 · today
- MoneyPrinterTurbo★ 136,645 · today