bpemb
Pre-trained subword embeddings in 275 languages, based on Byte-Pair Encoding.
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
GITHUB · launched Nov 5, 2026 · 50 upvotes (top 65%) · 0 views
Niche: machine learning developer tools · 303 launches, 11 in the last 7 days ▼ -45% · Explore this niche →
Category and tags
- Horizontal
- Model & infra
- Developer
- AI feature
- Hobby / open-source project
- byte-pair-encoding
- embeddings
- multilingual
- nlp
- python
- word-embeddings
Visit website ↗ Source listing ↗
More like this: all 303 launches in machine learning developer tools →
Could you build this?
Partial Running the BPE tokenization and embedding pipeline is scriptable, but producing good embeddings across 275 languages needs NLP and ML expertise plus substantial multilingual corpus processing.
What it would actually take: Reproducing it means collecting and cleaning Wikipedia-scale text for each language, training byte-pair encodings and then embeddings such as GloVe or word2vec, and packaging the results. The hard parts are data quality for low-resource languages and evaluating the embeddings. This requires NLP research experience and meaningful compute.
Competitors
Other products that read as similar to this one : 1023 launches clear the similarity bar, closest 8 shown.
Attention rank: #620 of 1024 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 372 days after the earliest competitor.
- orukeet · github · 2026-09-09 · 40 upvotes · similarity 0.57
- Linguin, a live multilingual dictionary for 120 languages · hn · 2026-07-20 · 5 upvotes · similarity 0.56
- QUAN-Transcribe · github · 2026-10-14 · 27 upvotes · similarity 0.54
- mlsubgen · github · 2026-10-01 · 13 upvotes · similarity 0.54
- pg_charabia · github · 2026-10-23 · 23 upvotes · similarity 0.53
- jevbench · github · 2026-09-22 · 7 upvotes · similarity 0.52
- mosaic · github · 2026-10-02 · 12 upvotes · similarity 0.52
- oido · github · 2026-09-30 · 85 upvotes · similarity 0.51
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.