ai agent evaluation platforms
Recent window: last 5.4 months (2026-05-01 → 2026-10-11), compared with the prior 5.4 months.
AI agent evaluation platforms provide automated testing, simulation, and benchmarking tools to assess the reliability, reasoning, and guardrails of autonomous systems. These solutions are built for software engineers and AI product teams deploying dynamic agents across complex workflows. Unlike standard LLM evaluation tools that score single-turn text prompts, they measure multi-step decision paths, tool usage, and environment interactions.
This theme's recent-window count is below the current min-count filter, so it does not appear in the opportunity scorecard.
Pulled from real Hacker News comment threads on this niche's launches.
Members
| Name | Source | Upvotes ▲ | Launched |
|---|---|---|---|
| awesome-harness-evolution | GITHUB | 23 | 2026-09-12 |
| AI SRE Arena, an Open Benchmark for AI SRE Agents on Kubernetes | HN | 24 | 2026-10-08 |
| QUALITY.md | HN | 29 | 2026-07-02 |
| misalignment | GITHUB | 30 | 2026-09-20 |
| rsi_eval | GITHUB | 31 | 2026-10-12 |
| Saffron: Find Your Next 10x Engineer | YC | 33 | 2026-04-27 |
| Magma - AI labs will pay you to learn from your agent's mistakes | YC | 37 | 2026-08-14 |
| Ax-check.com | HN | 37 | 2026-09-17 |
| Cua-Bench | HN | 40 | 2026-01-26 |
| Polymath | YC | 43 | 2026-02-26 |
| ShareCube | PH | 62 | 2026-10-01 |
| openeval | GITHUB | 68 | 2026-09-11 |
| QAgent | PH | 76 | 2026-09-17 |
| Agent-skills-eval | HN | 79 | 2026-05-07 |
| jevals | GITHUB | 83 | 2026-09-20 |
| laya-vs-jev | GITHUB | 100 | 2026-09-21 |
| jev-eval-agent | GITHUB | 101 | 2026-09-17 |
| AgentScore | PH | 151 | 2026-09-23 |
| laya-playground | GITHUB | 153 | 2026-09-20 |
| NovaSynth by Noveum | PH | 175 | 2026-09-17 |
| Fabraix | PH | 196 | 2026-05-08 |
| Lemma: Continuous Learning for AI Agents | YC | 204 | 2025-11-05 |
| JevHarness | GITHUB | 251 | 2026-09-21 |
| Agnost AI | PH | 289 | 2026-08-25 |
| oqoqo | PH | 339 | 2026-08-10 |
| iFixAi | PH | 345 | 2026-09-29 |
| Agent Arena | PH | 354 | 2026-06-26 |
| Cekura | PH | 367 | 2026-07-28 |
| AgentX | PH | 523 | 2026-06-22 |
| Prefactor | PH | 583 | 2026-07-28 |
| Plurai | PH | 757 | 2026-04-29 |
| unreal-agent | GITHUB | 1922 | 2026-09-21 |