I scraped 3B Goodreads reviews to train a better recommendation model
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
Hi everyone,For the past couple months I've been working on a website with two main features:- https://book.sv - put in a list of books and get recommendations on what to read next from a model trained on over a billion reviews- https://book.sv/intersect - put in a list of books and find the users on Goodreads who have read them all (if you don't want to be included in these results, you can opt-out here: https://book.sv/remove-my-data)Technical info available here: https://book.sv/how-it-worksNote 1: If you only provide one or two books, the model doesn't have a lot to work with and may include a handful of somewhat unrelated popular books in the results. If you want recommendations based on just one book, click the "Similar" button next to the book after adding it to the input book list on the recommendations page.Note 2: This is uncommon, but if you get an unexpected non-English titled book in the results, it is probably not a mistake and it very likely has an English edition. The "canonical" edition of a book I use for display is whatever one is the most popular, which is usually the English version, but this is not the case for all books, especially those by famous French or Russian authors.
HN · launched Nov 5, 2025 · 606 upvotes (top 1%) · 0 views
Niche: recommendation engines · 48 launches, 4 in the last 7 days · Explore this niche →
Category and tags
- Media & entertainment
- Search & retrieval
- B2C
- AI feature
- Hobby / open-source project
Visit website ↗ Source listing ↗
More like this: all 48 launches in recommendation engines →
Could you build this?
Partial The web interface and vector search endpoints are easily vibe-coded, but scraping 3 billion Goodreads reviews and training a collaborative filtering or embedding model at that scale requires massive data pipelines.
What it would actually take: Building this requires a distributed web scraping cluster running across thousands of residential proxies to bypass Goodreads/Amazon rate limits and Cloudflare protections for billions of pages. Ingesting and cleaning terabytes of raw text demands distributed data processing frameworks like Apache Spark or Ray. Finally, training modern matrix factorization or graph neural network recommendation models (e.g., LightGCN, two-tower models) on billions of interactions requires specialized ML systems engineering and high-memory GPU/TPU instances.
Discussion
20 comments analyzed.
Competitors mentioned: Netflix Cinematch (SVD-based recommendation algorithm), Netflix Prize winning algorithm
Concerns raised: Missing negative feedback/low ratings in model training, Firefox AbortError with concurrency timeout issues, Too many repetitive recommendations from same series/author, Profile access issues with large shelves (5000+ books), User confusion about input format (URL vs numeric ID)
Feature requests: Filter recommendations to exclude already-read authors, Click-through recommendations (chain recommendations from output books), Include negative ratings (1-2 stars) as model features, Reduce author/series repetition in results
Competitors
Other products that read as similar to this one : 221 launches clear the similarity bar, closest 8 shown.
Attention rank: #4 of 222 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Looks like the first mover among its competitors.
- See what readers who loved your favorite book/author also loved to read · hn · 2025-12-29 · 134 upvotes · similarity 0.57
- Visualizing How Books Reference Each Other Across 3k Years · hn · 2026-02-11 · 5 upvotes · similarity 0.55
- Extension to See Rating from Google Book, Amazon,StoryGraph on Goodread · hn · 2026-03-21 · 5 upvotes · similarity 0.52
- Zenòdot · hn · 2026-03-09 · 15 upvotes · similarity 0.50
- HackerBooks · hn · 2026-11-01 · 1857 upvotes · similarity 0.48
- I used Claude Code to discover connections between 100 books · hn · 2026-01-10 · 524 upvotes · similarity 0.46
- Off Your Shelf · ph · 2026-11-04 · 2 upvotes · similarity 0.46
- Bookrank · ph · 2026-09-11 · 1 upvotes · similarity 0.46
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.