Llmbuffer
Python library for cache-optimized LLM conversation history
Details
- External ID
- 48483607
- Source
- HN
- Company
- —
- Product
- Llmbuffer
- Website domain
- github.com
- Launched
- June 10, 2026
- Cohort
- —
- Upvotes
- 7
- Upvotes percentile
- 0.42008196721311475
- Tags
- —
- Fetched at
- Sept. 7, 2026, 9:26 p.m.
- Updated at
- Sept. 7, 2026, 9:26 p.m.
Description
I was not getting good cache utilization when including dynamic context in agent threads. After a lot of experimentation, I found a good pattern that minimizes how often long lived conversation history gets modified while still supporting dynamic context. It has flexible hooks for doing things like truncating or summarizing tool outputs when transitioning messages to the long term history. And I'm seeing >>90% of tokens hitting the cache for my agents despite including a lot of dynamic user context.There are a wide range of agent prompting strategies so I'd love to hear where this library works well and where there are patterns that don't fit well into the current API!
Enrichment
- Theme
- browser automation and scraping for AI
- Vertical
- Horizontal
- Function
- Dev tools
- Audience
- Developer
- AI stance
- AI feature
- Project type
- Hobby / open-source project
- Normalized one-liner
- cache optimization library for llm conversations
- Manually corrected
- False
Could you build this?
Yes It is a lightweight Python utility library designed to structure message arrays and manage prompt prefixes to maximize LLM KV-cache reuse.
Discussion
1 comment analyzed.
Concerns raised: Order missing first tool set definition
Competitors
Other products that read as similar to this one — 115 launches clear the similarity bar, closest 8 shown.
Attention rank: #63 of 116 (itself plus its competitors, highest first — normalized so YC and Product Hunt are compared fairly).
Launched 216 days after the earliest competitor.
- Librarian · hn · 2026-02-26 · 8 upvotes · similarity 0.50
- Context-compact · hn · 2026-03-07 · 6 upvotes · similarity 0.48
- UltraContext · hn · 2026-01-21 · 21 upvotes · similarity 0.48
- An LLM response cache that's aware of dynamic data · hn · 2026-01-07 · 17 upvotes · similarity 0.48
- ChatIndex · hn · 2025-11-26 · 17 upvotes · similarity 0.46
- Unify memory across agents and improve context rot, written in Rust · hn · 2026-04-05 · 5 upvotes · similarity 0.45
- Agent-cache · hn · 2026-04-16 · 18 upvotes · similarity 0.44
- An agent that tunes its own cache · hn · 2026-05-08 · 7 upvotes · similarity 0.44
Other launches for this product
- No other launches for this product.
Same idea, different domain
Nobody's really built a dev tools tool for Sales yet.