Maple-Preview
Ternary 20B MoE running at 120 tok/s on a iPhone
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
HN · launched Aug 4, 2026 · 173 upvotes (top 3%) · 0 views
Niche: llm inference engines · 133 launches, 4 in the last 7 days ▼ -33% · Explore this niche →
Category and tags
- Horizontal
- Model & infra
- Developer
- AI-native
- Commercial product
Visit website ↗ Source listing ↗
More like this: all 133 launches in llm inference engines →
Could you build this?
No Running a 20B parameter ternary Mixture-of-Experts (MoE) LLM at 120 tokens/second on an iPhone requires novel research in ternary quantization and custom low-level Apple Metal/ANE kernel engineering.
What it would actually take: Requires an inference engine written in C++/Metal utilizing customized 1.58-bit (ternary) matrix multiplication kernels tailored specifically for Apple Silicon GPU and Neural Engine registers. The hard parts are novel model weight quantization without catastrophic perplexity loss, efficient dynamic routing of MoE sparse experts within mobile memory limits, and hand-tuned SIMD vector instructions for ternary arithmetic.
Discussion
20 comments analyzed.
Competitors mentioned: Qwen 35B-A3B, Llama 3.5 35B, Llama 3.6 35B, Binary Bonsai 27B, Ternary Bonsai 27B
Concerns raised: Models don't know when to search internet, think they already know answers, Can't compress human knowledge into few bits without significant quality loss, Knowledge accuracy degrades fastest with quantization, Small models generate surface-level answers, not reliably accurate facts, App size (5.9 GB) too large for iOS, would trigger jetsam on most iPhones
Feature requests: Add tool/API integration for web search capability, Enable local Wikipedia or knowledge base querying via embedding search, Support for vision capabilities (VLM not text-only), Better reasoning and tool-use for conversational requirements
Competitors
Other products that read as similar to this one : 392 launches clear the similarity bar, closest 8 shown.
Attention rank: #25 of 393 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 269 days after the earliest competitor.
- Maple · hn · 2026-11-08 · 21 upvotes · similarity 0.53
- Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone · hn · 2026-08-03 · 312 upvotes · similarity 0.51
- nemotron-30b-cpu-256k · github · 2026-10-29 · 71 upvotes · similarity 0.46
- Qwen3.6-35B-A3B on a 16 GB M1 Pro with SSD-streamed MoE · hn · 2026-07-17 · 24 upvotes · similarity 0.44
- bonsai-apple-silicon · github · 2026-09-18 · 24 upvotes · similarity 0.43
- Snapintosh · github · 2026-11-08 · 10 upvotes · similarity 0.43
- aiistream-q3.6 · github · 2026-09-29 · 11 upvotes · similarity 0.43
- emu · github · 2026-09-25 · 34 upvotes · similarity 0.43
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.