TurboGPT: train 22KiB transformer in 13s
Details
- External ID
- 49898931
- Source
- HN
- Company
- —
- Product
- TurboGPT: train 22KiB transformer in 13s
- Website domain
- github.com
- Launched
- Sept. 29, 2026
- Cohort
- —
- Upvotes
- 53
- Upvotes percentile
- 0.8596491228070176
- Tags
- —
- Fetched at
- Oct. 1, 2026, 1:01 a.m.
- Updated at
- Oct. 1, 2026, 1:01 a.m.
Description
Inspired by minGPT for home experiments.Requires CUDA 13.4; build script is Windows only.
Enrichment
- Theme
- gpu compute and acceleration tools
- Vertical
- Horizontal
- Function
- Dev tools
- Audience
- Developer
- AI stance
- AI-native
- Project type
- Hobby / open-source project
- Normalized one-liner
- fast transformer training tool for developers
- Manually corrected
- False
Could you build this?
Partial While basic transformer models are well documented, writing high-performance, low-level custom CUDA kernels that train a micro-transformer in seconds requires specialized GPU programming expertise.
What it would actually take: The project is implemented in C++/CUDA and compiled with MSVC/NVCC for Windows, optimizing memory layout, kernel fusions, and matrix multiplications directly without heavy PyTorch runtime overhead. The hard part is low-level GPU memory alignment, warp-level primitives, and numerical stability in tight CUDA kernels to achieve 13-second training times. It requires an engineer experienced in low-level CUDA optimization and computer architecture.
Discussion
8 comments analyzed.
Competitors mentioned: Andrej Karpathy's GPT tutorial
Concerns raised: Reinventing the wheel without learning value, Low-effort AI/vibe-coded project, Labeling model size by disk instead of parameter count, Missing hn1g.txt file
Competitors
Other products that read as similar to this one — 83 launches clear the similarity bar, closest 8 shown.
Attention rank: #18 of 84 (itself plus its competitors, highest first — normalized so YC and Product Hunt are compared fairly).
Launched 329 days after the earliest competitor.
- TRiP · hn · 2026-04-30 · 38 upvotes · similarity 0.47
- I built GPT from scratch to understand how it works · hn · 2026-01-14 · 7 upvotes · similarity 0.44
- TinyGPU · github · 2026-09-19 · 22 upvotes · similarity 0.43
- Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU · hn · 2026-02-21 · 395 upvotes · similarity 0.42
- RUDA · github · 2026-09-09 · 14 upvotes · similarity 0.40
- Making a GBA game with GPT-6 Astra · hn · 2026-09-08 · 16 upvotes · similarity 0.40
- CUDA-for-AMD-Windows · github · 2026-09-13 · 230 upvotes · similarity 0.39
- dexgpt · github · 2026-09-09 · 26 upvotes · similarity 0.39
Other launches for this product
- No other launches for this product.
Same idea, different domain
Nobody's really built a dev tools tool for Sales yet.