genpark-robots-txt-crawl-delay-policy-evaluator-skill
RFC 9309 robots.txt parser with path permissions, Disallow/Allow rules, and crawl delay compliance
Details
- External ID
- 1394931809
- Source
- GITHUB
- Company
- —
- Product
- genpark-robots-txt-crawl-delay-policy-evaluator-skill
- Website domain
- github.com
- Launched
- Sept. 29, 2026
- Cohort
- —
- Upvotes
- 7
- Upvotes percentile
- 0.05976172175249808
- Tags
- agent-skills, agentic-browsing, compliance, crawl-delay, developer-tools, mcp, python-standard-library, rfc-9309, robots-txt, scraping-policy, web-crawler
- Fetched at
- Sept. 30, 2026, 1:02 a.m.
- Updated at
- Sept. 30, 2026, 1:02 a.m.
Enrichment
- Theme
- modular ai agent skills and toolkits
- Vertical
- Horizontal
- Function
- Dev tools
- Audience
- Developer
- AI stance
- Not AI
- Project type
- Hobby / open-source project
- Normalized one-liner
- robots.txt parser and crawl delay policy evaluator
- Manually corrected
- False
Could you build this?
Yes RFC 9309 specifies an explicit, deterministic grammar and path matching logic for robots.txt files, which an LLM can easily implement and test against standard test suites.
Competitors
Other products that read as similar to this one — 179 launches clear the similarity bar, closest 8 shown.
Attention rank: #62 of 180 (itself plus its competitors, highest first — normalized so YC and Product Hunt are compared fairly).
Launched 180 days after the earliest competitor.
- ToolsVale · ph · 2026-09-17 · 1 upvotes · similarity 0.47
- genpark-agent-sandboxed-file-diff-safety-auditor-skill · github · 2026-09-26 · 7 upvotes · similarity 0.47
- genpark-agent-sandboxed-file-diff-safety-auditor-skill · github · 2026-09-26 · 7 upvotes · similarity 0.47
- genpark-covert-data-exfiltration-firewall-skill · github · 2026-09-26 · 7 upvotes · similarity 0.47
- genpark-covert-data-exfiltration-firewall-skill · github · 2026-09-26 · 7 upvotes · similarity 0.47
- genpark-agent-tool-permission-rbac-validator-skill · github · 2026-09-26 · 7 upvotes · similarity 0.46
- genpark-agent-tool-permission-rbac-validator-skill · github · 2026-09-26 · 7 upvotes · similarity 0.46
- genpark-mcp-resource-uri-template-resolver-skill · github · 2026-09-29 · 7 upvotes · similarity 0.44
Other launches for this product
- No other launches for this product.
Same idea, different domain
Nobody's really built a dev tools tool for Sales yet.