nemotron-30b-cpu-256k
♟️ llama-modchip v2.1: Run 30B Sparse MoE LLMs on Pure CPU (0 GPUs) at 20–60.4 tok/s with a 256K Context Window via a 1.64MB Stockfish-PVS + RouteCrack L3 Drafter, 2MB HugePages & AVX-512 VNNI
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Visit website ↗ Source listing ↗
More like this: all 124 launches in llm inference engines →
GITHUB · launched Oct 29, 2026 · 71 upvotes (top 23%)
- Horizontal
- Model & infra
- Developer
- AI-native
- Hobby / open-source project
- apple-silicon
- avx512
- claude-code
- cpu-inference
- gguf
- hpc
- +8 more
Similar launches
See all 997 similar launches →Could you build this?
No Running a 30B MoE model on CPU at 30-50+ tok/s with a 256K context requires custom inference kernels, cache-aware memory layout and hugepage tuning. That is specialized systems and ML-kernel research.
What it would actually take: The stack would be a C/C++ or Rust inference engine with hand-tuned AVX-512 kernels, quantized weight repacking, state-space (Mamba-2) layer implementations and NUMA/cache-aware scheduling. It needs engineers who understand CPU microarchitecture and LLM internals, plus substantial benchmarking hardware.
Competitors
Other products that read as similar to this one : 997 launches clear the similarity bar, closest 8 shown.
Attention rank: #233 of 998 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 357 days after the earliest competitor.
- Nemotron 3.5 Bolt · ph · 2026-10-17 · 11 upvotes · similarity 0.54
- Full graphical desktop running on a 128MB VPS Alpine+XRDP+WindowMaker · hn · 2026-03-26 · 9 upvotes · similarity 0.49
- Long term Memory and 50M token window for LLM · hn · 2026-10-09 · 6 upvotes · similarity 0.49
- cyber-on-strata · github · 2026-10-09 · 17 upvotes · similarity 0.48
- local-llm-tuning-guide · github · 2026-10-16 · 21 upvotes · similarity 0.46
- I run 30B 22tok/s, 109tok/s not novel,6GB/16GB RAM overcoming llama.cpp · hn · 2026-07-29 · 5 upvotes · similarity 0.46
- local-llm-performance-profile-switcher · github · 2026-10-19 · 20 upvotes · similarity 0.46
- local-llm-vram-budget-optimizer · github · 2026-10-19 · 20 upvotes · similarity 0.46
Other launches for this product
- No other launches for this product.