I built GPT from scratch to understand how it works
Details
- External ID
- 46613808
- Source
- HN
- Company
- —
- Product
- I built GPT from scratch to understand how it works
- Website domain
- github.io
- Launched
- Jan. 14, 2026
- Cohort
- —
- Upvotes
- 7
- Upvotes percentile
- 0.3544137022397892
- Tags
- —
- Fetched at
- Sept. 7, 2026, 9:25 p.m.
- Updated at
- Sept. 7, 2026, 9:25 p.m.
Description
If you’ve ever been curious about how GPT actually works under the hood, I built a small project you might find interesting. I implemented a GPT-style transformer from scratch in a single notebook—covering tokenization, embeddings, causal self-attention, training, and autoregressive text generation without relying on high-level abstractions. The focus was on mechanistic clarity rather than scale or performance, and the notebook is structured to read more like a technical walkthrough than an experiment log. Feedback from people who’ve built or studied transformers would be very welcome.
Enrichment
- Theme
- multimodal generative ai and developer tools
- Vertical
- Horizontal
- Function
- Dev tools
- Audience
- Developer
- AI stance
- Not AI
- Project type
- Hobby / open-source project
- Normalized one-liner
- gpt implementation from scratch
- Manually corrected
- False
Could you build this?
Yes The author built an educational nanoGPT-style transformer in a single Python notebook using PyTorch. Implementing basic causal self-attention, tokenization, and forward passes is a well-trodden tutorial path that AI assistants can generate comprehensively.
Discussion
No comments on this launch.
Competitors
Other products that read as similar to this one — 76 launches clear the similarity bar, closest 8 shown.
Attention rank: #51 of 77 (itself plus its competitors, highest first — normalized so YC and Product Hunt are compared fairly).
Launched 75 days after the earliest competitor.
- Transformer Primitives · hn · 2026-06-23 · 7 upvotes · similarity 0.63
- TurboGPT: train 22KiB transformer in 13s · hn · 2026-09-29 · 53 upvotes · similarity 0.44
- NanoEuler · hn · 2026-06-19 · 6 upvotes · similarity 0.43
- NanoEuler · hn · 2026-06-28 · 55 upvotes · similarity 0.43
- Interactive 3D and 2D Visualization of GPT-2 · hn · 2026-03-19 · 12 upvotes · similarity 0.42
- I built a lite LPU that can do inference on Karpathy's MicroGPT · hn · 2026-08-24 · 18 upvotes · similarity 0.41
- GPT Image 2.5 · ph · 2026-09-11 · 1 upvotes · similarity 0.41
- GPT Image 2.5 AI Video Generator · ph · 2026-09-06 · 1 upvotes · similarity 0.40
Other launches for this product
- No other launches for this product.
Same idea, different domain
Nobody's really built a dev tools tool for Sales yet.