Llmbuffer
Python library for cache-optimized LLM conversation history
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
I was not getting good cache utilization when including dynamic context in agent threads. After a lot of experimentation, I found a good pattern that minimizes how often long lived conversation history gets modified while still supporting dynamic context. It has flexible hooks for doing things like truncating or summarizing tool outputs when transitioning messages to the long term history. And I'm seeing >>90% of tokens hitting the cache for my agents despite including a lot of dynamic user context.There are a wide range of agent prompting strategies so I'd love to hear where this library works well and where there are patterns that don't fit well into the current API!
HN · launched Jun 10, 2026 · 7 upvotes (top 58%) · 0 views
Niche: llm optimization tools · 127 launches, 9 in the last 7 days ▲ +50% · Explore this niche →
Category and tags
- Horizontal
- Dev tools
- Developer
- AI feature
- Hobby / open-source project
Visit website ↗ Source listing ↗
More like this: all 127 launches in llm optimization tools →
Could you build this?
Yes It is a lightweight Python utility library designed to structure message arrays and manage prompt prefixes to maximize LLM KV-cache reuse.
Discussion
1 comment analyzed.
Concerns raised: Order missing first tool set definition
Competitors
Other products that read as similar to this one : 243 launches clear the similarity bar, closest 8 shown.
Attention rank: #140 of 244 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 223 days after the earliest competitor.
- Librarian · hn · 2026-02-26 · 8 upvotes · similarity 0.52
- UltraContext · hn · 2026-01-21 · 21 upvotes · similarity 0.50
- Context-compact · hn · 2026-03-07 · 6 upvotes · similarity 0.50
- An LLM response cache that's aware of dynamic data · hn · 2026-01-07 · 17 upvotes · similarity 0.50
- ChatIndex · hn · 2025-11-26 · 17 upvotes · similarity 0.48
- Unify memory across agents and improve context rot, written in Rust · hn · 2026-04-05 · 5 upvotes · similarity 0.47
- Sulci · ph · 2026-10-29 · 2 upvotes · similarity 0.47
- Agent-cache · hn · 2026-04-16 · 18 upvotes · similarity 0.46
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.