NanoRL
RL training for LLMs in ~1,800 lines
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
The smallest async RL trainer I could write: one loop that runs REINFORCE on CartPole on a laptop and async GRPO on a cluster (e.g. 8xH100 trainer, 8 vLLM workers, ran as a [SkyPilot job group](https://docs.skypilot.ai/en/latest/examples/job-groups.html) on k8s ).All without Ray or TRL or DeepSpeed etc., workers talk to the trainer over stdlib HTTP.
HN · launched Aug 13, 2026 · 11 upvotes (top 39%) · 0 views
Niche: reinforcement learning frameworks · 70 launches, 0 in the last 7 days · Explore this niche →
Category and tags
- Horizontal
- Model & infra
- Developer
- AI-native
- Hobby / open-source project
Visit website ↗ Source listing ↗
More like this: all 70 launches in reinforcement learning frameworks →
Could you build this?
No Implementing a distributed asynchronous RL trainer (GRPO/REINFORCE) that orchestrates GPU clusters and inference engines without existing frameworks requires deep expertise in distributed ML systems and RL mathematics.
What it would actually take: The stack involves PyTorch, custom CUDA/C++ extensions or optimized Triton kernels, raw socket/NCCL communication or custom async queues, and direct integration with vLLM worker processes. The hard part is managing distributed model weights synchronization, asynchronous off-policy gradient corrections, memory management during rollout generation, and low-latency cluster scheduling across H100 nodes without high-level abstractions like Ray or DeepSpeed.
Discussion
No comments on this launch.
Competitors
Other products that read as similar to this one : 35 launches clear the similarity bar, closest 8 shown.
Attention rank: #14 of 36 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 285 days after the earliest competitor.
- I built a tiny LLM to demystify how language models work · hn · 2026-04-06 · 915 upvotes · similarity 0.40
- Mini-vLLM in ~500 lines of Python · hn · 2025-12-28 · 5 upvotes · similarity 0.39
- Minimal LLM Post-Training Experiments on an 8GB GPU (SFT, DPO, GRPO) · hn · 2026-08-01 · 21 upvotes · similarity 0.37
- Run 500B+ Parameter LLMs Locally on a Mac Mini · hn · 2026-03-09 · 17 upvotes · similarity 0.37
- I RL-trained an agent that trains models with RL (for ~$1.3k) · hn · 2026-07-14 · 107 upvotes · similarity 0.36
- GLM Mini · ph · 2026-09-14 · 1 upvotes · similarity 0.35
- awesome-post-training-RL · github · 2026-09-11 · 11 upvotes · similarity 0.35
- A tiny LLM running at 21,000 tok/s on a $250 FPGA (Live Demo) · hn · 2026-08-10 · 79 upvotes · similarity 0.35
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.