llm benchmarking tools
Recent window: last 5.4 months (2026-05-01 → 2026-10-12), compared with the prior 5.4 months.
LLM benchmarking tools evaluate, compare, and rank the performance of large language models across diverse tasks, benchmarks, and hardware setups. They are built for AI engineers, data scientists, and technical teams looking to select the most capable and cost-effective models for production. Unlike general AI observability or prompt management platforms, these tools focus specifically on standardized testing suites, head-to-head arenas, and repeatable evaluation metrics.
Pulled from real Hacker News comment threads on this niche's launches.
Members
| Name | Source | Upvotes ▼ | Launched |
|---|---|---|---|
| cheatbench | GITHUB | 8 | 2026-09-15 |
| I took the calculator away from 50 LLMs and graded the arithmetic | HN | 7 | 2026-08-12 |
| opus-5.5-benchmarks | GITHUB | 7 | 2026-09-23 |
| Do Codex skills save tokens? A six-run task-size benchmark | HN | 7 | 2026-08-03 |
| Redactle LLM Leaderboard | HN | 5 | 2026-09-03 |
| LLMadness | HN | 5 | 2026-03-19 |
| A game/benchmark where AI bots hunt each other | HN | 5 | 2026-01-08 |
| Pencil Puzzle Bench | HN | 5 | 2026-03-03 |
| Peer Arena | HN | 5 | 2025-12-28 |
| SOTA long memory eval with open source models | HN | 5 | 2026-03-03 |
| We benchmarked 18 LLMs on OCR (7K+ calls) | HN | 5 | 2026-04-22 |
| LLM Skirmish | HN | 5 | 2026-02-04 |
| New eval from SWE-bench team evalutes LMs based on goals not tickets | HN | 5 | 2025-11-05 |
| Best Fit | PH | 4 | 2026-09-30 |
| nerfWatch() | PH | 4 | 2026-09-26 |
| BestLLMfor | PH | 2 | 2026-09-09 |
| NerfTracker | PH | 2 | 2026-09-30 |
| jev-bench | PH | 2 | 2026-09-28 |
| State of AI Recommendations | PH | 1 | 2026-10-04 |
| SkyBreak Platform | PH | 1 | 2026-10-14 |
| Rubbletag | PH | 1 | 2026-10-02 |
| TaskBill — Cost-Per-Task Benchmark Kit | PH | 1 | 2026-10-07 |