Llama 3.2 3B and Keiro Research achieves 85% on SimpleQA
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
ran this over the weekend. stack was Llama 3.2 3B running locally + Keiro Research API for retrieval.85.0% on 4,326 questions. where that lands:ROMA (357B): 93.9% OpenDeepSearch (671B): 88.3% Sonar Pro: 85.8% Llama 3.2 3B + Keiro: 85.0%the systems ahead of us are running models 100-200x larger. that's why they're ahead. not better retrieval, not better prompting — just way more parameters.the interesting part is how small the gap is despite that. 3 points behind a 671B model. 0.8 behind Sonar Pro. at some point you have to ask what you're actually buying with all that compute for this class of task.Want to know how low the reader model can go before it starts mattering. in this setup it clearly wasn't the limiting factor and also if smaller models with web enabled will perform as good( if not better) as larger models for a lot of non coding tasksFull benchmark script + results --> https://github.com/h-a-r-s-h-s-r-a-h/benchmarkKeiro research -- https://www.keirolabs.cloud/docs/api-reference/research
HN · launched Mar 7, 2026 · 6 upvotes (top 71%) · 0 views
Niche: large language models · 191 launches, 4 in the last 7 days ▼ -33% · Explore this niche →
Category and tags
- Horizontal
- Search & retrieval
- Developer
- AI-native
- Commercial product
Visit website ↗ Source listing ↗
More like this: all 191 launches in large language models →
Could you build this?
Partial Running a local Llama 3.2 3B model and querying an API is simple, but achieving an 85% score on SimpleQA relies heavily on proprietary high-precision retrieval infrastructure (Keiro Research).
What it would actually take: Replicating this benchmark score requires building an advanced web search, indexing, and fact-checking retrieval engine comparable to Keiro or Perplexity. The backend necessitates low-latency web crawlers, state-of-the-art embedding and reranking models, semantic chunking, and hallucination-minimizing RAG pipelines before feeding context into the small local LLM.
Discussion
1 comment analyzed.
Competitors
Other products that read as similar to this one : 101 launches clear the similarity bar, closest 8 shown.
Attention rank: #77 of 102 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 124 days after the earliest competitor.
- Omni · hn · 2026-06-05 · 6 upvotes · similarity 0.42
- Nimble · ph · 2026-10-14 · 80 upvotes · similarity 0.41
- PhAIL · hn · 2026-03-31 · 21 upvotes · similarity 0.39
- A new benchmark for testing LLMs for deterministic outputs · hn · 2026-04-29 · 60 upvotes · similarity 0.39
- What is HN thinking? Real-time sentiment and concept analysis · hn · 2026-02-12 · 37 upvotes · similarity 0.39
- open-medical-jev · github · 2026-09-25 · 38 upvotes · similarity 0.39
- Fast NF4 dequantization Triton kernel (1.41x faster than bitsandbytes) · hn · 2026-07-15 · 5 upvotes · similarity 0.38
- Llm.sql · hn · 2026-04-24 · 8 upvotes · similarity 0.37
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.