Scrawly
Free open-source SEO crawler for Google and AI search
Get picks like this daily. The day's top launches, AI/tech news, and a weekly opportunity spotlight — straight to your inbox.
This is 1 of 183 launches in seo audit and tracking software — see how it stacks up on momentum and crowding →
49 other launches read as similar to this one →
Details
- External ID
- 1273067
- Source
- PH
- Company
- —
- Product
- Scrawly
- Website domain
- producthunt.com
- Launched
- Oct. 12, 2026
- Cohort
- —
- Upvotes
- 1
- Upvotes percentile
- 0.33
- Tags
- Open Source, SEO, Developer Tools, GitHub
Description
Scrawly is a free, open-source desktop SEO crawler for macOS, Windows and Linux. It crawls your site, renders JavaScript and runs 250 technical SEO, GEO and AI-search checks, then tells you in plain English what to fix. Unlike Screaming Frog or Sitebulb, there's no licence, no account and no page cap. It also tests whether GPTBot, ClaudeBot and PerplexityBot can reach you, audits llms.txt, applies reversible WordPress fixes and ships an MCP server. MIT licensed.
Enrichment
- Niche
- seo audit and tracking software
- Vertical
- Marketing
- Function
- Analytics & BI
- Audience
- Developer
- AI stance
- Not AI
- Project type
- Hobby / open-source project
- Normalized one-liner
- open-source web crawler for seo and ai search optimization
- Manually corrected
- False
Could you build this?
Partial Basic web scraping is trivial, but building a desktop SEO crawler that renders headless JavaScript at scale without memory leaks and evaluates 250 deep technical audits requires complex systems engineering.
What it would actually take: The app needs an Electron or Tauri shell running a pool of headless browser instances (Playwright/Puppeteer/Chromium) with an embedded high-performance database (SQLite/DuckDB) to handle hundreds of thousands of URLs without exhausting system memory. The hard part is managing crawler memory management, anti-bot handling, resource pooling for headless JS rendering, and codifying hundreds of complex SEO heuristics.
Competitors
Other products that read as similar to this one — 49 launches clear the similarity bar, closest 8 shown.
Attention rank: #15 of 50 (itself plus its competitors, highest first — normalized so YC and Product Hunt are compared fairly).
Launched 287 days after the earliest competitor.
- scary_spider_seo · ph · 2026-09-21 · 1 upvotes · similarity 0.49
- Crawlie · hn · 2026-06-18 · 20 upvotes · similarity 0.44
- SEOSage · ph · 2026-10-08 · 12 upvotes · similarity 0.41
- SiteAud!tLint · ph · 2026-10-02 · 1 upvotes · similarity 0.41
- DIY SEO Hub · ph · 2026-09-18 · 2 upvotes · similarity 0.40
- RankRoot · ph · 2026-10-03 · 1 upvotes · similarity 0.39
- Links Frog · ph · 2026-10-02 · 1 upvotes · similarity 0.38
- Blake SEO Tools · ph · 2026-10-02 · 1 upvotes · similarity 0.37
Other launches for this product
- No other launches for this product.