Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
HN · launched Mar 14, 2026 · 408 upvotes (top 1%) · 0 views
Niche: digital history and reference archives · 150 launches, 3 in the last 7 days ▼ -80% · Explore this niche →
Category and tags
- Horizontal
- Data infrastructure
- Developer
- Not AI
- Hobby / open-source project
Visit website ↗ Source listing ↗
More like this: all 150 launches in digital history and reference archives →
Could you build this?
Yes This is a simple scheduled script (cron/GitHub Actions) polling the public Hacker News Firebase API and writing batches to Parquet files uploaded to Hugging Face.
Discussion
20 comments analyzed.
Competitors mentioned: DuckDB with SQLite extension, HuggingFace hub, Microsoft.AI.Extensions (.NET), Alternative HN frontends
Concerns raised: AI slop flooding makes pre-2020 content preservation harder, Legal permissions unclear for using HN data, Ingestion time too slow (week-long process), Rate limiting and API access constraints
Feature requests: DuckDB to SQLite conversion (reduce ingestion to 5 minutes), Batch abstraction for chat/embedding interfaces, Near real-time data refresh capability, C# / .NET equivalent implementation
Competitors
Other products that read as similar to this one : 408 launches clear the similarity bar, closest 8 shown.
Attention rank: #11 of 409 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 133 days after the earliest competitor.
- Hackerbrief · hn · 2026-03-16 · 82 upvotes · similarity 0.60
- Hacker News RSS Feed Directory · hn · 2026-04-04 · 8 upvotes · similarity 0.56
- Hacker News with reduced priority for AI driven content · hn · 2026-08-06 · 6 upvotes · similarity 0.52
- Hacker News with reduced priority for AI driven content · hn · 2026-08-05 · 24 upvotes · similarity 0.52
- Hacker News, but every headline is hysterical clickbait · hn · 2025-12-19 · 188 upvotes · similarity 0.51
- The Front Page · hn · 2026-05-24 · 29 upvotes · similarity 0.51
- Hacker News but for comments posted on HN · hn · 2026-11-02 · 19 upvotes · similarity 0.50
- The Hacker News Puzzle · hn · 2026-08-18 · 5 upvotes · similarity 0.50
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.