Betelgeuse
A custom fork for frontier-scale mixture-of-experts inference on an RTX 3060 with 12 GB VRAM, 16 GB system RAM and local NVMe.** Betelgeuse keeps dense computation on the GPU and brings routed experts into a bounded VRAM pool only when needed.
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
GITHUB · launched Oct 31, 2026 · 90 upvotes (top 15%) · 0 views
Niche: llm inference engines · 133 launches, 4 in the last 7 days ▼ -33% · Explore this niche →
Category and tags
- Horizontal
- Model & infra
- Developer
- AI-native
- Hobby / open-source project
Visit website ↗ Source listing ↗
More like this: all 133 launches in llm inference engines →
Could you build this?
No This is a custom inference-engine fork that streams routed MoE experts between NVMe, system RAM, and VRAM with bounded memory on a single consumer GPU. It requires low-level CUDA kernel, memory-scheduling, and model-internals expertise that vibe coding does not supply.
What it would actually take: The stack is C++/CUDA with a modified inference runtime (e.g., on top of llama.cpp or similar), custom prefetch and caching logic for expert weights, and profiling tooling. The hard parts are overlapping NVMe-to-GPU transfers with compute, predicting which experts to load, and handling the memory budget without stalls. This requires systems and GPU performance engineers with deep familiarity with transformer inference internals.
Competitors
Other products that read as similar to this one : 969 launches clear the similarity bar, closest 8 shown.
Attention rank: #171 of 970 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 358 days after the earliest competitor.
- Infernix · github · 2026-10-08 · 98 upvotes · similarity 0.59
- saber-engine · github · 2026-11-05 · 59 upvotes · similarity 0.56
- OpenRelay: The Inference Delivery Network · yc · 2026-08-07 · 13 upvotes · similarity 0.55
- talus-public · github · 2026-10-28 · 30 upvotes · similarity 0.55
- TandemLLM · github · 2026-09-28 · 21 upvotes · similarity 0.55
- GPUWorks · hn · 2026-11-04 · 30 upvotes · similarity 0.54
- gr8 · github · 2026-10-10 · 27 upvotes · similarity 0.52
- nCompass - Optimize performance on GPUs, 10x faster · yc · 2026-03-02 · 19 upvotes · similarity 0.52
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.