Every number in a tiny GPT, from one forward pass to one step of RLHF
Get picks like this daily. The day's top launches, AI/tech news, and a weekly opportunity spotlight — straight to your inbox.
This is 1 of 27 launches in technical data visualization tools — see how it stacks up on momentum and crowding →
83 other launches read as similar to this one →
Details
- External ID
- 49991390
- Source
- HN
- Company
- —
- Product
- Every number in a tiny GPT, from one forward pass to one step of RLHF
- Website domain
- manogya.dev
- Launched
- Oct. 7, 2026
- Cohort
- —
- Upvotes
- 8
- Upvotes percentile
- 0.46
- Tags
- —
Description
Source: https://github.com/manogyasingh/transformer-visualisation Was taking a course on reasoning LMs with a 50% project component. Shortly after I started, I realised I didn't fully understand how these work. I know the architecture and the theory and steps but still there are stages where either the purpose, or the functioning was unclear to me and so I wanted an interactive walkthrough of one token's generation. So here it is, made with Opus 5.5.It's a step by step walkthrough of a toy GPT-2 like model. Every matrix fits on screen, every number is computed live in the browser, and hovering over any cell shows the exact math of that step. It covers inference -- following the generation of one token from tokensiation to sampling, one training step of pretraining, and one PPO iteration of RLHF for post-training.Inspired in part by Brendan Bycroft's LLM visualization (https://bbycroft.net/llm)
Enrichment
- Niche
- technical data visualization tools
- Vertical
- Education
- Function
- Dev tools
- Audience
- Developer
- AI stance
- AI-native
- Project type
- Hobby / open-source project
- Normalized one-liner
- interactive visualization of internal numbers in a tiny gpt
- Manually corrected
- False
Could you build this?
Partial The frontend presentation can be built with React, but numerically calculating every single activation and weight through a forward pass and RLHF step requires deep knowledge of transformer mechanics.
What it would actually take: Building this requires writing an exact, step-by-step PyTorch or pure NumPy implementation of a minimal GPT model (attention matrices, feed-forward, loss, policy gradient/PPO for RLHF) and serializing every intermediate numerical state into a structured JSON. The web frontend then synchronizes interactive walkthrough state with the mathematical trace.
Discussion
No comments on this launch.
Competitors
Other products that read as similar to this one — 83 launches clear the similarity bar, closest 8 shown.
Attention rank: #48 of 84 (itself plus its competitors, highest first — normalized so YC and Product Hunt are compared fairly).
Launched 331 days after the earliest competitor.
- I built GPT from scratch to understand how it works · hn · 2026-01-14 · 7 upvotes · similarity 0.50
- Transformer Primitives · hn · 2026-06-23 · 7 upvotes · similarity 0.47
- Symbolic Circuit Distillation: prove program to LLM circuit equivalence · hn · 2026-01-06 · 16 upvotes · similarity 0.42
- Duplicate 3 layers in a 24B LLM, logical deduction .22→.76. No training · hn · 2026-03-18 · 265 upvotes · similarity 0.42
- Interactive 3D and 2D Visualization of GPT-2 · hn · 2026-03-19 · 12 upvotes · similarity 0.42
- Zero-power photonic language model–code · hn · 2025-11-29 · 18 upvotes · similarity 0.41
- NanoEuler · hn · 2026-06-19 · 6 upvotes · similarity 0.41
- NanoEuler · hn · 2026-06-28 · 55 upvotes · similarity 0.41
Other launches for this product
- No other launches for this product.