Keep large tool output out of LLM context: 3x accuracy 95% fewer tokens
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
LLM agents often place raw JSON tool outputs directly in the prompt. After a few tool calls, earlier results get compacted or truncated and answers become incorrect or inconsistent.I built Sift, a drop-in MCP gateway that stores tool outputs as local artifacts (filesystem blobs indexed in SQLite) and returns an `artifact_id` plus compact schema hints when responses are large or paginated.Instead of reasoning over full JSON in the prompt, the model runs a small Python query: def run(data, schema, params): return max(data, key=lambda x: x["magnitude"])["place"] Query code runs in a constrained subprocess (AST/import guards + timeout/memory caps). Only the computed result is returned to the model.Benchmark (Claude Sonnet 4.6, 103 questions across 12 datasets):- Baseline (raw JSON in prompt): 34/103 (33%), 10.7M input tokens- Sift (artifact + code query): 102/103 (99%), 489K input tokensOpen benchmark + MIT code: https://github.com/lourencomaciel/sift-gatewayInstall: pipx install sift-gateway sift-gateway init --from claude Works with Claude Code, Cursor, Windsurf, Zed, and VS Code. Existing MCP servers and tools require no changes.
HN · launched Mar 5, 2026 · 10 upvotes (top 42%) · 0 views
Niche: llm optimization tools · 129 launches, 6 in the last 7 days ▼ -14% · Explore this niche →
Category and tags
- Horizontal
- Dev tools
- Developer
- AI-native
- Hobby / open-source project
Visit website ↗ Source listing ↗
More like this: all 129 launches in llm optimization tools →
Could you build this?
Yes It is a lightweight MCP proxy server that intercepts JSON payloads, writes them to local disk/SQLite, and returns reference IDs, which is well-suited for AI-assisted development.
Discussion
1 comment analyzed.
Concerns raised: Context compaction issues with tool-heavy agents
Competitors
Other products that read as similar to this one : 128 launches clear the similarity bar, closest 8 shown.
Attention rank: #60 of 129 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 124 days after the earliest competitor.
- Calling tools w/ Python improves LLM perf. vs MCP (77.1% on BrowseComp) · hn · 2025-12-16 · 10 upvotes · similarity 0.47
- A new benchmark for testing LLMs for deterministic outputs · hn · 2026-04-29 · 60 upvotes · similarity 0.45
- Mcp2cli · hn · 2026-03-09 · 146 upvotes · similarity 0.44
- Rust-split · hn · 2026-09-12 · 5 upvotes · similarity 0.42
- Reducing LLM input tokens by 70% · hn · 2026-05-12 · 56 upvotes · similarity 0.40
- Prompt-refiner · hn · 2025-12-17 · 7 upvotes · similarity 0.40
- Model Tools Protocol (MTP) · hn · 2026-02-10 · 9 upvotes · similarity 0.40
- Context Gateway · hn · 2026-03-13 · 97 upvotes · similarity 0.40
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.