Nicheloom visits: 64,306 all-time · 8,841 in the last 7 days

The opportunity tracker for new startups.

Today: 1,283 launches · Niche of the day: browser-based pdf tools · See what launched today →

KradleBench

Multi-agent AI evals in Minecraft

Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.

Want to reach these readers? Sponsor an issue.

Hi HN -This is James and Kemal. In past lives we led Firebase and Tensorflow/JAX respectively. HN has been a particularly big part of my (James’) career and the trajectory I’ve taken would be meaningfully different without it. Thank you!Our team shipped a benchmark: KradleBench. We test AI in Minecraft Worlds.1. The worlds are multi-agent. Humans and other AIs are in there too, often pursuing their own goals. Collaboration and conflict skills fall out of this property.2. The world are dynamic. They constantly change, which means that agents need to react. “Capture the Flag” has a single winning condition, and many different ways to achieve it. This leads to many emergent and creative strategies.3. The worlds are real-time. They are not turn-based, they do not wait for AIs to finish thinking. Nature balanced speed and intelligence in humans, we can understand this in AIs too.Underneath all this is Kradle, a platform to test and understand intelligence. It’s a petri dish where any intelligence type (LLM, World Model, Human) can be compared and generate useful information to improve current systems, whether it’s on capabilities or safety.Some findings so far: - A skilled human was #1, until yesterday when Astra 6.2 surpassed them. We lose badly on solo tasks and out-compete on multi-agent ones - Astra 6.2 iterates on its strategy 4x faster than Fable 5.2. The Claude family tends to think hard and stick with strategies. The OpenAI models tend to be more reactive and iterative, which serves them well in multi-agent, dynamic, real-time evals.We’ve got videos on all the top model’s page, and we’ll be producing them when a new model launches. Sign up on the site to get them and for when the platform opens.Thanks for reading,James & Kemal

HN · launched Nov 18, 2026 · 8 upvotes (top 50%) · 0 views

Niche: ai agent evaluation platforms · 190 launches, 10 in the last 7 days ▼ -9% · Explore this niche →

Category and tags

  • Horizontal
  • Observability & eval
  • Developer
  • AI-native
  • Commercial product

Visit website ↗ Source listing ↗

Could you build this?

No This is a research benchmark that evaluates frontier models in multi-agent Minecraft environments, with human baselines and scoring methodology. It requires custom game-environment infrastructure, large model-API budgets, and novel evaluation design from ML researchers.

What it would actually take: A real version needs a headless Minecraft simulation harness with deterministic environments, agent-to-game tooling, and a scoring framework across collaboration, economy, conflict, logic, and creativity. Running many frontier models at scale is expensive, and validity depends on careful methodology and human baselines. The hard parts are research design and infrastructure, not the web frontend.

Discussion

7 comments analyzed.

Competitors mentioned: Opus, Fable

Concerns raised: poor conversational interaction with humans, over-fixates on details, weaker for coding than rivals, poor online reputation

Competitors

Other products that read as similar to this one : 1817 launches clear the similarity bar, closest 8 shown.

Attention rank: #876 of 1818 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).

Launched 385 days after the earliest competitor.

The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.