Nicheloom visits: 58,094 all-time · 7,637 in the last 7 days

The opportunity tracker for new startups.

Today: 1,953 launches · Niche of the day: specialized calculation tools · See what launched today →

Web Harvester

Go scraper: smart proxies, health checks, rate limits

Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.

Want to reach these readers? Sponsor an issue.

Web Harvester is an open-source, provider-agnostic web collection engine built in Go. It combines health-aware proxy rotation, authenticated proxies, adaptive rate limiting, cooldowns, runtime configuration reloads, Prometheus metrics, and Redis Streams. Built for resilient, high-throughput pipelines, it includes reproducible benchmarks for measuring performance and reliability. Clone the repo, run the benchmarks, inspect the code, and help build better web data infrastructure.

PH · launched Oct 23, 2026 · 2 upvotes (top 22%) · 0 views

Niche: web scraping tools and apis · 172 launches, 12 in the last 7 days ▲ +9% · Explore this niche →

Category and tags
  • Horizontal
  • Data infrastructure
  • Developer
  • Not AI
  • Hobby / open-source project
  • Productivity
  • Open Source
  • Developer Tools
  • GitHub

Visit website ↗ Source listing ↗

More like this: all 172 launches in web scraping tools and apis →

Could you build this?

Yes It is an engineering project in Go with proxy rotation, rate limiting, metrics, and Redis queues. These are known patterns that an experienced builder assisted by AI can assemble.

Competitors

Other products that read as similar to this one : 124 launches clear the similarity bar, closest 8 shown.

Attention rank: #14 of 125 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).

Launched 346 days after the earliest competitor.

  • Proxy Tester by ScrapeOps · ph · 2026-08-09 · 208 upvotes · similarity 0.56
  • Scrappy · hn · 2026-02-16 · 10 upvotes · similarity 0.49
  • DatafetchPro Tools · ph · 2026-10-19 · 1 upvotes · similarity 0.46
  • Gouger · ph · 2026-10-22 · 1 upvotes · similarity 0.45
  • SCRAPR · ph · 2026-03-09 · 259 upvotes · similarity 0.45
  • ProxyLister · ph · 2026-11-03 · 1 upvotes · similarity 0.44
  • DataHarvester · ph · 2026-11-06 · 1 upvotes · similarity 0.42
  • proxy-benchmark · ph · 2026-09-18 · 1 upvotes · similarity 0.42

Related data

The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.