doc2harness
Official Code for the paper "Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents "
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Visit website ↗ Source listing ↗
GITHUB · launched Oct 11, 2026 · 33 upvotes (top 39%)
- agent
- benchmark
- harness
- llm
- llm-agents
Similar launches
See all 55 similar launches →Could you build this?
Partial The benchmark harness, containers, and scoring are vibe-codeable, but the curated test suite and running many frontier models are costly and need research-level design choices.
What it would actually take: Build a containerized runner that gives a model a spec and a skeleton, freezes its output, and drives it with a scripted mock model while recording requests, file diffs, exit codes, and command output. The hard part is designing 328 test cases with positive controls that avoid vacuous passes, which requires research judgment. Running 19 models with three builds each also needs significant API budget and compute.
Competitors
Other products that read as similar to this one : 55 launches clear the similarity bar, closest 8 shown.
Attention rank: #21 of 56 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 345 days after the earliest competitor.
- docjev · github · 2026-09-19 · 384 upvotes · similarity 0.52
- alexharonyandega-dev · github · 2026-09-22 · 17 upvotes · similarity 0.45
- JevAgentHarness · github · 2026-10-15 · 24 upvotes · similarity 0.43
- OpenVLHarness · github · 2026-10-15 · 14 upvotes · similarity 0.42
- DRsubmission · github · 2026-10-19 · 20 upvotes · similarity 0.40
- json2dir-bimbo · github · 2026-10-06 · 23 upvotes · similarity 0.40
- bsuir-word-docs · github · 2026-09-29 · 12 upvotes · similarity 0.39
- A free DOCX editor with MCP server for editing · hn · 2026-08-08 · 14 upvotes · similarity 0.37
Other launches for this product
- No other launches for this product.
More details
- External ID
- 1414201248
- Company
- n/a
- Product
- doc2harness
- Website domain
- geyuyao.com
- Cohort
- n/a
- Upvotes percentile
- 0.61
- All tags
- agent, benchmark, harness, llm, llm-agents
- Manually corrected
- No