Agent-skills-eval
Test whether Agent Skills improve outputs
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
HN · launched May 7, 2026 · 79 upvotes (top 12%) · 0 views
Niche: ai agent evaluation platforms · 182 launches, 8 in the last 7 days ▼ -50% · Explore this niche →
Category and tags
- Horizontal
- Observability & eval
- Developer
- AI-native
- Hobby / open-source project
Visit website ↗ Source listing ↗
More like this: all 182 launches in ai agent evaluation platforms →
Could you build this?
Yes Evaluation suites for agent prompts and skills consist of test runners, LLM API calls, scoring functions, and report generation that AI assistants generate proficiently.
Discussion
20 comments analyzed.
Competitors mentioned: Codex, Claude (for code execution), tidewave (Rails MCP server for DB queries)
Concerns raised: Skills are often ignored by models even when properly defined, Token cost per run not reported alongside correctness metrics, Skills may not outperform fine-tuning given model behavior, Prompts not tightly coupled with capabilities, making skills unreliable, Models default to shell habits over skill instructions
Feature requests: Add token cost per run and cost-benefit analysis to reports, Support skill comparison benchmarks beyond with/without evals, Include time, pass rate, and estimated cost in evaluation reports, Add token count consumed per skill and detailed aggregate stats, Implement self-reflection pass to identify skill improvement recommendations
Competitors
Other products that read as similar to this one : 3486 launches clear the similarity bar, closest 8 shown.
Attention rank: #393 of 3487 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 190 days after the earliest competitor.
- Skill-up · hn · 2026-07-30 · 5 upvotes · similarity 0.84
- agent-skills · github · 2026-09-15 · 16 upvotes · similarity 0.77
- research_skills · github · 2026-10-26 · 36 upvotes · similarity 0.72
- better-skill · github · 2026-10-27 · 84 upvotes · similarity 0.70
- skills · github · 2026-10-08 · 67 upvotes · similarity 0.70
- skills · github · 2026-10-05 · 19 upvotes · similarity 0.69
- agent-reliability-skills · github · 2026-10-30 · 34 upvotes · similarity 0.68
- skill-audit · github · 2026-09-09 · 22 upvotes · similarity 0.68
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.