RepoGym
Validate the task before you benchmark the coding agent
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
Turn repository tasks into repeatable coding-agent evaluations. Define a task, check that doing nothing fails and the reference fix passes, then inspect scores and trajectories. Includes Python and JavaScript examples, composable graders, task mining and HTML reports. Free, MIT-licensed Python library. Start from the source checkout; local execution is for trusted code.
PH · launched Sep 23, 2026 · 1 upvote (top 69%) · 0 views
Niche: ai agent evaluation platforms · 182 launches, 8 in the last 7 days ▼ -50% · Explore this niche →
Category and tags
- Horizontal
- Observability & eval
- Developer
- AI-native
- Commercial product
- Developer Tools
- GitHub
Visit website ↗ Source listing ↗
More like this: all 182 launches in ai agent evaluation platforms →
Could you build this?
Yes It is an open-source Python evaluation harness that runs test suites inside Docker containers and reports diff/pass/fail results for AI coding agents.
Competitors
Other products that read as similar to this one : 681 launches clear the similarity bar, closest 8 shown.
Attention rank: #495 of 682 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 327 days after the earliest competitor.
- Agentic Workbench · ph · 2026-10-20 · 2 upvotes · similarity 0.53
- RepoGauntlet · ph · 2026-09-28 · 1 upvotes · similarity 0.53
- calcifier · github · 2026-10-14 · 9 upvotes · similarity 0.50
- reuF2F · github · 2026-10-18 · 22 upvotes · similarity 0.49
- Co-Review · ph · 2026-11-08 · 2 upvotes · similarity 0.48
- Cheddar-bench · hn · 2026-02-22 · 9 upvotes · similarity 0.48
- skills · github · 2026-10-05 · 19 upvotes · similarity 0.48
- Score your GitHub repo for AI coding agents · hn · 2026-03-17 · 7 upvotes · similarity 0.48
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.