LLM-Gateway
Zero-Trust LLM Gateway
Details
- External ID
- 47542999
- Source
- HN
- Company
- —
- Product
- LLM-Gateway
- Website domain
- github.com
- Launched
- March 27, 2026
- Cohort
- —
- Upvotes
- 7
- Upvotes percentile
- 0.4108241082410824
- Tags
- —
- Fetched at
- Sept. 7, 2026, 9:26 p.m.
- Updated at
- Sept. 7, 2026, 9:26 p.m.
Description
I built an OpenAI-compatible LLM gateway that routes requests to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang... anything that speaks /v1/chat/completions. Single Go binary, one YAML config file, no infrastructure.It does the things you'd expect from this kind of gateway... semantic routing via a three-layer cascade (keyword heuristics, embedding similarity, LLM classifier) that picks the best model when clients omit the model field, weighted round-robin load balancing across local inference servers with health checks and failover.The part I think is most interesting is the network layer. The gateway and backends communicate over zrok/OpenZiti overlay networks... reach a GPU box behind NAT, expose the gateway to clients, put components anywhere with internet connectivity behind firewalls... no port forwarding, no VPN. Zero-trust in both directions. Most LLM proxies solve the API translation problem. This one also solves the network problem.Apache 2.0. https://github.com/openziti/llm-gatewayI work for NetFoundry, which sponsors the OpenZiti project this is built on.
Enrichment
- Theme
- Vertical
- Security
- Function
- Model & infra
- Audience
- B2B
- AI stance
- AI feature
- Project type
- Commercial product
- Normalized one-liner
- zero-trust gateway for llm access
- Manually corrected
- False
Could you build this?
Yes It is a single-binary reverse proxy in Go translating OpenAI-compatible REST schemas and routing requests to different LLM backends based on simple YAML rules.
Discussion
1 comment analyzed.
Competitors mentioned: Anthropic, OpenAI, Google, Open source models
Concerns raised: OSS models require expertise to evaluate and select, Need to write custom evaluations for OSS models, Unclear quality/consistency from OSS providers vs. established APIs, Complexity overhead outweighs cost savings of OSS
Competitors
Other products that read as similar to this one — 224 launches clear the similarity bar, closest 8 shown.
Attention rank: #138 of 225 (itself plus its competitors, highest first — normalized so YC and Product Hunt are compared fairly).
Launched 140 days after the earliest competitor.
- llm-gateway-comparison · github · 2026-09-17 · 59 upvotes · similarity 0.53
- MCP Gateway · hn · 2025-12-03 · 10 upvotes · similarity 0.51
- Millwright · hn · 2026-07-22 · 10 upvotes · similarity 0.51
- llm-api-gateway-free-tier-fr · github · 2026-09-29 · 9 upvotes · similarity 0.49
- llm-api-proxy-no-vpn-fr · github · 2026-09-29 · 16 upvotes · similarity 0.48
- llm-api-gateway-ru · github · 2026-09-28 · 15 upvotes · similarity 0.47
- llm-api-proxy-no-vpn-ja · github · 2026-09-29 · 15 upvotes · similarity 0.47
- ngrok AI Gateway · ph · 2026-08-05 · 304 upvotes · similarity 0.46
Other launches for this product
- No other launches for this product.
Same idea, different domain
Nobody's really built a model & infra tool for Fintech yet.