LLM Skirmish
a benchmark where LLMs play RTS games, by writing code
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
I wanted to create an LLM game benchmark that put this generation of frontier LLMs' top skill, coding, on full display.Ten years ago, a team released a game called Screeps. It was described as an "MMO RTS sandbox for programmers." In Screeps, human players write javascript strategies that get executed in the game's environment.The Screeps paradigm, writing code and having it execute in a real-time game environment, is well suited for an LLM benchmark. Drawing on a version of the Screeps open source API, LLM Skirmish pits LLMs head-to-head in a series of 1v1 real-time strategy games.
HN · launched Feb 4, 2026 · 5 upvotes (top 89%) · 0 views
Niche: llm benchmarking tools · 84 launches, 1 in the last 7 days · Explore this niche →
Category and tags
- Horizontal
- Observability & eval
- Developer
- AI-native
- Hobby / open-source project
Visit website ↗ Source listing ↗
More like this: all 84 launches in llm benchmarking tools →
Could you build this?
Yes Building a benchmark harness around an existing open-source game engine (Screeps API) to feed game state to LLMs and evaluate their output scripts in 1v1 matches is readily vibe-coded with standard scripts and web frontends.
Discussion
2 comments analyzed.
Feature requests: Extend to general-soldier models and agent swarms, Apply to broader model evaluation beyond code tasks
Competitors
Other products that read as similar to this one : 111 launches clear the similarity bar, closest 8 shown.
Attention rank: #106 of 112 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 91 days after the earliest competitor.
- A real-time strategy game that AI agents can play · hn · 2026-02-25 · 220 upvotes · similarity 0.83
- LLMPvP · ph · 2026-09-29 · 2 upvotes · similarity 0.46
- Can you beat frontier LLMs at social strategy games? | Multi-Agent Arena by Olam Labs, evaluating through multi-agent simulations · yc · 2026-08-05 · 9 upvotes · similarity 0.44
- Gemini 4 Argon plays StarCraft · hn · 2026-10-16 · 254 upvotes · similarity 0.42
- I taught LLMs to play Magic: The Gathering against each other · hn · 2026-02-17 · 117 upvotes · similarity 0.42
- Watch LLMs play 21,000 hands of Poker · hn · 2026-01-08 · 36 upvotes · similarity 0.41
- Speed Miners · hn · 2026-01-17 · 49 upvotes · similarity 0.40
- Schnyders Dice Of Fire · ph · 2026-10-19 · 1 upvotes · similarity 0.40
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.