ai agent evaluation platforms
Recent window: last 5.4 months (2026-05-01 → 2026-10-11), compared with the prior 5.4 months.
AI agent evaluation platforms provide automated testing, simulation, and benchmarking tools to assess the reliability, reasoning, and guardrails of autonomous systems. These solutions are built for software engineers and AI product teams deploying dynamic agents across complex workflows. Unlike standard LLM evaluation tools that score single-turn text prompts, they measure multi-step decision paths, tool usage, and environment interactions.
This theme's recent-window count is below the current min-count filter, so it does not appear in the opportunity scorecard.
Pulled from real Hacker News comment threads on this niche's launches.
Members
| Name | Source | Upvotes ▼ | Launched |
|---|---|---|---|
| unreal-agent | GITHUB | 1922 | 2026-09-21 |
| Plurai | PH | 757 | 2026-04-29 |
| Prefactor | PH | 583 | 2026-07-28 |
| AgentX | PH | 523 | 2026-06-22 |
| Cekura | PH | 367 | 2026-07-28 |
| Agent Arena | PH | 354 | 2026-06-26 |
| iFixAi | PH | 345 | 2026-09-29 |
| oqoqo | PH | 339 | 2026-08-10 |
| Agnost AI | PH | 289 | 2026-08-25 |
| JevHarness | GITHUB | 251 | 2026-09-21 |
| Lemma: Continuous Learning for AI Agents | YC | 204 | 2025-11-05 |
| Fabraix | PH | 196 | 2026-05-08 |
| NovaSynth by Noveum | PH | 175 | 2026-09-17 |
| laya-playground | GITHUB | 153 | 2026-09-20 |
| AgentScore | PH | 151 | 2026-09-23 |
| jev-eval-agent | GITHUB | 101 | 2026-09-17 |
| laya-vs-jev | GITHUB | 100 | 2026-09-21 |
| jevals | GITHUB | 83 | 2026-09-20 |
| Agent-skills-eval | HN | 79 | 2026-05-07 |
| QAgent | PH | 76 | 2026-09-17 |
| openeval | GITHUB | 68 | 2026-09-11 |
| ShareCube | PH | 62 | 2026-10-01 |
| Polymath | YC | 43 | 2026-02-26 |
| Cua-Bench | HN | 40 | 2026-01-26 |
| Ax-check.com | HN | 37 | 2026-09-17 |
| Magma - AI labs will pay you to learn from your agent's mistakes | YC | 37 | 2026-08-14 |
| Saffron: Find Your Next 10x Engineer | YC | 33 | 2026-04-27 |
| rsi_eval | GITHUB | 31 | 2026-10-12 |
| misalignment | GITHUB | 30 | 2026-09-20 |
| QUALITY.md | HN | 29 | 2026-07-02 |
| AI SRE Arena, an Open Benchmark for AI SRE Agents on Kubernetes | HN | 24 | 2026-10-08 |
| awesome-harness-evolution | GITHUB | 23 | 2026-09-12 |
| MinigamesBenchmark | GITHUB | 23 | 2026-10-06 |
| skill-audit | GITHUB | 22 | 2026-09-09 |
| DORA | GITHUB | 22 | 2026-09-25 |
| DataFlow-AgentMM | GITHUB | 21 | 2026-09-22 |
| Part_HackBench | GITHUB | 21 | 2026-09-10 |
| Nyx | HN | 20 | 2026-04-19 |
| sac-agent-skill-assurance | GITHUB | 20 | 2026-09-14 |
| MaLiang-Harness | GITHUB | 20 | 2026-09-27 |
| mini-harness | GITHUB | 19 | 2026-09-13 |
| AAArena | GITHUB | 19 | 2026-10-06 |
| llm-multi-agent-game-evaluation | GITHUB | 19 | 2026-09-23 |
| acceptance | GITHUB | 18 | 2026-09-21 |
| Open-Source Agentic QA Harness with Memory | HN | 18 | 2026-05-20 |
| minesweeper | GITHUB | 18 | 2026-09-21 |
| Open-source simulation testing infra for voice agents | HN | 16 | 2026-09-10 |
| WideSWE | GITHUB | 16 | 2026-09-27 |
| agent-duel | GITHUB | 16 | 2026-10-05 |
| BLINDSPOT | GITHUB | 15 | 2026-09-14 |