Watch LLMs play 21,000 hands of Poker
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
PokerBench is my attempt at a new LLM benchmark wherein frontier models play Texas Hold'em in an arena setting. It also features a simulator to view individual games and observe how the different models reason about poker strategy. Opus/Haiku, Gemini Pro/Flash, GPT-5.2/5 mini, and Grok 4.1 Fast Reasoning have all been included.All code -> https://github.com/JoeAzar/pokerbench
HN · launched Jan 8, 2026 · 36 upvotes (top 26%) · 0 views
Niche: llm benchmarking tools · 84 launches, 1 in the last 7 days · Explore this niche →
Category and tags
- Media & entertainment
- Observability & eval
- B2C
- AI-native
- Hobby / open-source project
Visit website ↗ Source listing ↗
More like this: all 84 launches in llm benchmarking tools →
Could you build this?
Yes A simulation harness and web visualizer where an open-source poker game engine drives turns, sends prompts to various LLM provider APIs, and logs the game state for playback.
Discussion
19 comments analyzed.
Competitors mentioned: Deterministic bot with probability tables, Trading benchmarks (Deepseek), Open source models
Concerns raised: Very expensive to run games ($30/game for 6-handed, $6/game for 4-handed), Sample size may be too small (160 games, ~21k hands), Results may be random walk, need reinitialization to test robustness, Win rate vs. profit inconsistency (high win rate models losing money)
Feature requests: Incorporate open source/open weights models, Add tool use incorporation, Explain why smaller models outperform larger ones, Run multiple simultaneous trials for robustness
Competitors
Other products that read as similar to this one : 117 launches clear the similarity bar, closest 8 shown.
Attention rank: #29 of 118 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 64 days after the earliest competitor.
- Play poker with LLMs, or watch them play against each other · hn · 2026-01-10 · 163 upvotes · similarity 0.65
- I taught LLMs to play Magic: The Gathering against each other · hn · 2026-02-17 · 117 upvotes · similarity 0.51
- governor-of-poker-game-enhancement · github · 2026-11-10 · 102 upvotes · similarity 0.50
- Poker Practice Simulator · ph · 2026-11-04 · 1 upvotes · similarity 0.50
- JevPokerBench · github · 2026-09-21 · 11 upvotes · similarity 0.50
- Can you beat frontier LLMs at social strategy games? | Multi-Agent Arena by Olam Labs, evaluating through multi-agent simulations · yc · 2026-08-05 · 9 upvotes · similarity 0.48
- Betsfans Ai Poker · ph · 2026-10-16 · 1 upvotes · similarity 0.48
- PokerGame · ph · 2026-09-18 · 3 upvotes · similarity 0.48
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.