Cachet
A drop-in semantic cache for LLM APIs, 100% local, in Rust
Details
- External ID
- 48643854
- Source
- HN
- Company
- —
- Product
- Cachet
- Website domain
- github.com
- Launched
- June 23, 2026
- Cohort
- —
- Upvotes
- 5
- Upvotes percentile
- 0.12568306010928962
- Tags
- —
- Fetched at
- Sept. 7, 2026, 9:26 p.m.
- Updated at
- Sept. 7, 2026, 9:26 p.m.
Enrichment
- Theme
- ML inference and model optimization
- Vertical
- Horizontal
- Function
- Model & infra
- Audience
- Developer
- AI stance
- AI feature
- Project type
- Commercial product
- Normalized one-liner
- semantic cache for llm api calls
- Manually corrected
- False
Could you build this?
Partial While an HTTP proxy that hashes and caches LLM outputs is simple, building an ultra-fast, local semantic cache in Rust requires embedding models, fast vector indexing, and approximate nearest neighbor search.
What it would actually take: The implementation relies on Rust (using Axum or Actix) embedding an ONNX runtime (via ort or candle) to run local embedding models (like all-MiniLM-L6-v2) alongside a vector index (like HNSW or USearch) and an embedded key-value store (like RocksDB or Sled). The difficult aspects are optimizing inference and cosine distance searches to achieve sub-millisecond overhead while preserving high semantic recall and managing memory bounds. It requires systems programming in Rust and familiarity with vector search internals.
Discussion
No comments on this launch.
Competitors
Other products that read as similar to this one — 1624 launches clear the similarity bar, closest 8 shown.
Attention rank: #1348 of 1625 (itself plus its competitors, highest first — normalized so YC and Product Hunt are compared fairly).
Launched 237 days after the earliest competitor.
- casita · github · 2026-09-27 · 71 upvotes · similarity 0.69
- Octopii, a runtime for writing distributed applications in Rust · hn · 2025-12-09 · 17 upvotes · similarity 0.68
- Mnemo · hn · 2026-06-03 · 60 upvotes · similarity 0.65
- loom · github · 2026-09-09 · 11 upvotes · similarity 0.63
- WolfStack · hn · 2026-03-08 · 32 upvotes · similarity 0.61
- typed-lm · github · 2026-09-25 · 9 upvotes · similarity 0.61
- Flux, A Python-like language in Rust to solve ML orchestration overhead · hn · 2026-01-24 · 5 upvotes · similarity 0.61
- genpark-agentic-cache-semantic-deduplicator-skill · github · 2026-09-14 · 8 upvotes · similarity 0.60
Other launches for this product
Same idea, different domain
Nobody's really built a model & infra tool for Fintech yet.