Agent-skills-eval
Test whether Agent Skills improve outputs
Details
- External ID
- 48046023
- Source
- HN
- Company
- —
- Product
- Agent-skills-eval
- Website domain
- github.com
- Launched
- May 7, 2026
- Cohort
- —
- Upvotes
- 79
- Upvotes percentile
- 0.8820678513731826
- Tags
- —
- Fetched at
- Sept. 7, 2026, 9:26 p.m.
- Updated at
- Sept. 7, 2026, 9:26 p.m.
Enrichment
- Theme
- ai agent infrastructure and tooling
- Vertical
- Horizontal
- Function
- Observability & eval
- Audience
- Developer
- AI stance
- AI-native
- Project type
- Commercial product
- Normalized one-liner
- evaluation tool for agent skills
- Manually corrected
- False
Could you build this?
Yes Evaluation suites for agent prompts and skills consist of test runners, LLM API calls, scoring functions, and report generation that AI assistants generate proficiently.
Discussion
20 comments analyzed.
Competitors mentioned: Codex, Claude (for code execution), tidewave (Rails MCP server for DB queries)
Concerns raised: Skills are often ignored by models even when properly defined, Token cost per run not reported alongside correctness metrics, Skills may not outperform fine-tuning given model behavior, Prompts not tightly coupled with capabilities, making skills unreliable, Models default to shell habits over skill instructions
Feature requests: Add token cost per run and cost-benefit analysis to reports, Support skill comparison benchmarks beyond with/without evals, Include time, pass rate, and estimated cost in evaluation reports, Add token count consumed per skill and detailed aggregate stats, Implement self-reflection pass to identify skill improvement recommendations
Competitors
Other products that read as similar to this one — 1639 launches clear the similarity bar, closest 8 shown.
Attention rank: #169 of 1640 (itself plus its competitors, highest first — normalized so YC and Product Hunt are compared fairly).
Launched 190 days after the earliest competitor.
- Skill-up · hn · 2026-07-30 · 5 upvotes · similarity 0.83
- agent-skills · github · 2026-09-15 · 16 upvotes · similarity 0.76
- skill-audit · github · 2026-09-09 · 22 upvotes · similarity 0.66
- Ingot, evidence-gated optimization and version control for agent skills · hn · 2026-07-22 · 7 upvotes · similarity 0.65
- SkillGraph · hn · 2025-11-12 · 19 upvotes · similarity 0.65
- nix-skills · github · 2026-09-22 · 8 upvotes · similarity 0.64
- skillbox · github · 2026-09-17 · 222 upvotes · similarity 0.63
- skills · github · 2026-09-13 · 6 upvotes · similarity 0.63
Other launches for this product
- No other launches for this product.
Same idea, different domain
Nobody's really built a observability & eval tool for Media & entertainment yet.