AegisCrawler
Record once, replay forever — a production-grade browser data-collection platform. A PageResearch Agent extension turns real interactions into humanized DSL rules, a Go server schedules lease-based tasks, and ScriptCat workers execute them in real browsers. Optional LLM rule enhancement with security scan + human diff review.
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
GITHUB · launched Sep 20, 2026 · 15 upvotes (top 52%) · 0 views
Niche: web scraping tools and apis · 172 launches, 12 in the last 7 days ▲ +9% · Explore this niche →
Category and tags
- Horizontal
- Workflow automation
- B2B
- AI feature
- Commercial product
- anthropic
- browser-automation
- browser-extension
- chrome-extension
- data-collection
- dsl
- +14 more
Visit website ↗ Source listing ↗
More like this: all 172 launches in web scraping tools and apis →
Could you build this?
Partial While the Go task scheduler and browser extension recorder are straightforward, building a reliable distributed crawler that uses real browsers, humanized DSL replays, and bypasses modern bot detection requires complex anti-detect engineering.
What it would actually take: The architecture comprises a Chrome extension to record DOM events into a custom DSL, a Go/PostgreSQL distributed worker queue managing task leases, and runner instances (via ScriptCat or Puppeteer/Playwright). The difficult engineering challenge lies in humanized action emulation (curved mouse paths, randomized timings, CDP fingerprint masking) to bypass Cloudflare and Akamai bot protections at production scale without constant manual intervention.
Competitors
Other products that read as similar to this one : 432 launches clear the similarity bar, closest 8 shown.
Attention rank: #210 of 433 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 319 days after the earliest competitor.
- resocket-Watchcat · github · 2026-11-05 · 57 upvotes · similarity 0.46
- Galdor · hn · 2026-06-13 · 7 upvotes · similarity 0.43
- Display.dev · hn · 2026-06-18 · 16 upvotes · similarity 0.43
- Oversteer - The Most Reliable Browser Agent for Any Web Task · yc · 2025-11-05 · 15 upvotes · similarity 0.43
- agweb · github · 2026-11-11 · 182 upvotes · similarity 0.42
- m-x-machina · github · 2026-10-04 · 9 upvotes · similarity 0.41
- sediment · github · 2026-09-21 · 8 upvotes · similarity 0.41
- ClawDesk · hn · 2026-03-31 · 5 upvotes · similarity 0.41
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.