L88
A Local RAG System on 8GB VRAM (Need Architecture Feedback)
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
Hey everyone,I’ve been working on a project called L88 — a local RAG system that I initially focused on UI/UX for, so the retrieval and model architecture still need proper refinement.Repo: https://github.com/Hundred-Trillion/L88-FullI’m running this on 8GB VRAM and a strong CPU (128GB RAM). Embeddings and preprocessing run on CPU, and the main model runs on GPU. One limitation I ran into is that my evaluator and generator LLM ended up being the same model due to compute constraints, which defeats the purpose of evaluation.I’d really appreciate feedback on:Better architecture ideas for small-VRAM RAGSplitting evaluator/generator roles effectivelyImproving the LangGraph pipelineAny bugs or design smells you noticeWays to optimize the system for local hardwareI’m 18 and still learning a lot about proper LLM architecture, so any technical critique or suggestions would help me grow as a developer. If you check out the repo or leave feedback, it would mean a lot — I’m trying to build a solid foundation and reputation through real projects.Thanks!
HN · launched Feb 24, 2026 · 12 upvotes (top 40%) · 0 views
Niche: rag developer frameworks and tools · 59 launches, 3 in the last 7 days · Explore this niche →
Category and tags
- Horizontal
- Search & retrieval
- Developer
- AI-native
- Hobby / open-source project
More like this: all 59 launches in rag developer frameworks and tools →
Could you build this?
Yes This is a local RAG interface built on standard open-source tools (like Ollama, LangChain/LlamaIndex, and local vector stores) constrained to run on 8GB VRAM.
Discussion
3 comments analyzed.
Competitors mentioned: Qdrant, FAISS, SQLite, Supabase (vector/storage solutions), BGE-base-en-v1.5, bge-reranker-v2-m3 (embedding/reranking models), Cross-encoder rerankers, BM25 (keyword search)
Concerns raised: VRAM/RAM constraints on free-tier/low-resource servers, Latency from multiple sequential LLM calls, Circular evaluation issue: using same model for retrieval and quality scoring, Vector search alone misses exact-match queries (titles, authors), Cross-encoder rerankers destroy performance on resource-constrained setups
Feature requests: Merge router/analyzer/rewriter into single LLM call to reduce latency, Implement hybrid BM25 + vector search with dynamic weighting by query type, Add semantic caching layer (cosine similarity threshold) alongside exact-match hashing, Use cross-encoder for evaluation/thresholding instead of LLM-as-judge, Parent-child chunking: embed small chunks, store large payloads in lightweight DB
Competitors
Other products that read as similar to this one : 426 launches clear the similarity bar, closest 8 shown.
Attention rank: #189 of 427 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 116 days after the earliest competitor.
- local-llm-tuning-guide · github · 2026-10-16 · 21 upvotes · similarity 0.52
- local-llm-vram-budget-optimizer · github · 2026-10-19 · 20 upvotes · similarity 0.51
- I wrote a custom assembler for CHIP-8 in C++ · hn · 2026-09-17 · 28 upvotes · similarity 0.51
- Shoehorn, a library to quantize an LLM to fit your Mac's VRAM · hn · 2026-08-14 · 6 upvotes · similarity 0.50
- E80: an 8-bit CPU in structural VHDL · hn · 2026-01-17 · 34 upvotes · similarity 0.50
- Minimal LLM Post-Training Experiments on an 8GB GPU (SFT, DPO, GRPO) · hn · 2026-08-01 · 21 upvotes · similarity 0.48
- You can now train your own LLMs with less memory, on your consumer GPUs · hn · 2026-11-10 · 5 upvotes · similarity 0.48
- VRAMGlass · ph · 2026-09-19 · 1 upvotes · similarity 0.47
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.