I tested 15 free AI models at building real software on a $25/year VPS
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
HN · launched Apr 2, 2026 · 37 upvotes (top 19%) · 1 view
Niche: ai benchmarking platforms · 147 launches, 4 in the last 7 days ▼ -43% · Explore this niche →
Category and tags
- Horizontal
- Observability & eval
- Developer
- AI-native
- Hobby / open-source project
Visit website ↗ Source listing ↗
More like this: all 147 launches in ai benchmarking platforms →
Could you build this?
Yes This is a benchmarking script and test harness written to run API prompts against multiple LLM endpoints, execute integration tests in a shell, and log output.
Discussion
7 comments analyzed.
Competitors mentioned: DigitalOcean, OpenRouter, OpenCode
Concerns raised: Opencode doesn't publish quantization details, VPS pricing claims seem inaccurate ($25/year unrealistic), Unclear what hardware models actually ran on, Missing model sizes and quantization specifications in comparisons, How top open models compare to SOTA paid models
Feature requests: Publish quantization specifications for models tested, Include model sizes in comparisons, Clarify hardware/infrastructure setup for reproducibility, Compare performance: VPS hardware vs laptop
Competitors
Other products that read as similar to this one : 1577 launches clear the similarity bar, closest 8 shown.
Attention rank: #267 of 1578 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 155 days after the earliest competitor.
- I used AI to built static recompilers for 5 retro game systems · hn · 2026-05-26 · 5 upvotes · similarity 0.53
- I built a museum of AI interfaces · hn · 2026-08-24 · 6 upvotes · similarity 0.52
- Belvedir: We autonomously make custom models so the world can have trillions of them · yc · 2026-08-11 · 64 upvotes · similarity 0.52
- Thomas: the first YC-backed AI founder · yc · 2026-06-19 · 16 upvotes · similarity 0.50
- Locus Founder · yc · 2026-06-14 · 7 upvotes · similarity 0.50
- Lumabri · hn · 2026-10-22 · 5 upvotes · similarity 0.50
- Archal: API sandboxes, built for AI agents · yc · 2026-08-22 · 61 upvotes · similarity 0.49
- Phantom · hn · 2026-03-30 · 18 upvotes · similarity 0.49
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.