Shoehorn, a library to quantize an LLM to fit your Mac's VRAM
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
I made this after seeing someone posit the idea online yesterday over lunch then spent some time refining it. So far it's pretty impressive IMO! Right now I am running Qwen3-30B-A3B on my 24gb unified memory m4 MacBook Pro at 50 tok/sec and this should definitely not be working for such a large model on my middling hardware.Things are detailed in the README to get up and running and DESIGN.md has details on all the choices and such made along the way.
HN · launched Aug 14, 2026 · 6 upvotes (top 69%) · 0 views
Niche: llm inference and deployment tools · 188 launches, 7 in the last 7 days ▲ +17% · Explore this niche →
Category and tags
- Horizontal
- Model & infra
- Developer
- AI-native
- Hobby / open-source project
Visit website ↗ Source listing ↗
More like this: all 188 launches in llm inference and deployment tools →
Could you build this?
Partial While wrapping existing quantization libraries (like llama.cpp / gguf / MLX) can be scripted, writing a tailored quantization and memory-profiling pipeline that dynamically fits large models into Apple Silicon unified memory at high inference speeds requires solid low-level ML engineering.
What it would actually take: A proper solution requires deep integration with Apple's Metal Performance Shaders (MPS) or MLX, writing custom quantization kernels (such as 2-bit to 4-bit AWQ or GGUF formats), and profiling unified memory bandwidth to dynamically adjust layer offloading and activation caches.
Discussion
No comments on this launch.
Competitors
Other products that read as similar to this one : 577 launches clear the similarity bar, closest 8 shown.
Attention rank: #415 of 578 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 287 days after the earliest competitor.
- Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s · hn · 2026-09-01 · 240 upvotes · similarity 0.57
- KV-psi, using Linux PSI to to trim an LLM KV cache · hn · 2026-06-27 · 8 upvotes · similarity 0.56
- You can now train your own LLMs with less memory, on your consumer GPUs · hn · 2026-11-10 · 5 upvotes · similarity 0.53
- IronMule · ph · 2026-09-17 · 1 upvotes · similarity 0.52
- I run 30B 22tok/s, 109tok/s not novel,6GB/16GB RAM overcoming llama.cpp · hn · 2026-07-29 · 5 upvotes · similarity 0.52
- L88 · hn · 2026-02-24 · 12 upvotes · similarity 0.50
- Mac LLM Calculator · ph · 2026-10-06 · 2 upvotes · similarity 0.50
- Run 500B+ Parameter LLMs Locally on a Mac Mini · hn · 2026-03-09 · 17 upvotes · similarity 0.49
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.