StreamHouse
S3-native Kafka alternative written in Rust
Details
- External ID
- 47146676
- Source
- HN
- Company
- —
- Product
- StreamHouse
- Website domain
- github.com
- Launched
- Feb. 25, 2026
- Cohort
- —
- Upvotes
- 10
- Upvotes percentile
- 0.5316711590296496
- Tags
- —
- Fetched at
- Sept. 7, 2026, 9:25 p.m.
- Updated at
- Sept. 7, 2026, 9:25 p.m.
Description
Hey HN,I built StreamHouse, an open-source streaming platform that replaces Kafka's broker-managed storage with direct S3 writes. The goal: same semantics, fraction of the cost.How it works: Producers batch and compress records, a stateless server manages partition routing and metadata (SQLite for dev, PostgreSQL for prod), and segments land directly in S3. Consumers read from S3 with a local segment cache. No broker disks to manage, no replication factor to tune — S3 gives you 11 nines of durability out of the box.What's there today: - Producer API with batching, LZ4 compression, and offset tracking (62K records/sec) - Consumer API with consumer groups, auto-commit, and multi-partition fanout (30K+ records/sec) - Kafka-compatible protocol (works with existing Kafka clients) - REST API, gRPC API, CLI, and a web UI - Docker Compose setup for trying it locally in 5 minutesThe cost model is what motivated this. Kafka's storage costs scale with replication factor × retention × volume. With S3 at $0.023/GB/month, storing a TB of events costs ~$23/month instead of hundreds on broker EBS volumes.Written in Rust, ~50K lines across 15 crates. Apache 2.0 licensed.GitHub: https://github.com/gbram1/streamhouseHappy to answer questions about the architecture, tradeoffs, or what I learned building this.
Enrichment
- Theme
- database infrastructure and developer tools
- Vertical
- —
- Function
- Data infrastructure
- Audience
- B2B
- AI stance
- Not AI
- Project type
- Commercial product
- Normalized one-liner
- s3-native kafka alternative
- Manually corrected
- False
Could you build this?
No Implementing a distributed streaming system with Kafka semantics directly on S3 demands deep expertise in distributed systems, consensus, storage tiering, zero-copy I/O, and Rust systems programming.
What it would actually take: A real implementation requires custom Rust storage engines using async runtimes (Tokio), optimized multipart S3 write batching, strict partitioned log semantics, and Raft/distributed consensus or strict distributed metadata locking. The core difficulty lies in achieving low consumer latency and high write throughput against S3's eventual consistency and object-creation latencies while guaranteeing exact-once or at-least-once delivery guarantees.
Discussion
8 comments analyzed.
Competitors mentioned: rustfs, minio, Kafka, S3
Concerns raised: quickstart has errors, requires API key setup, minio no longer maintained, higher latency than alternatives
Competitors
Other products that read as similar to this one — 95 launches clear the similarity bar, closest 8 shown.
Attention rank: #62 of 96 (itself plus its competitors, highest first — normalized so YC and Product Hunt are compared fairly).
Launched 116 days after the earliest competitor.
- BlazeMQ · hn · 2026-02-10 · 5 upvotes · similarity 0.51
- S2-lite, an open source Stream Store · hn · 2026-01-21 · 77 upvotes · similarity 0.51
- Streambed · hn · 2026-05-31 · 129 upvotes · similarity 0.50
- UnisonDB · hn · 2025-11-01 · 17 upvotes · similarity 0.49
- DuckDB for Kafka Stream Processing · hn · 2025-12-08 · 77 upvotes · similarity 0.48
- Sofka · hn · 2026-08-19 · 5 upvotes · similarity 0.46
- Durable Streams · hn · 2025-12-09 · 10 upvotes · similarity 0.46
- PicoMQ · hn · 2026-08-24 · 159 upvotes · similarity 0.46
Other launches for this product
- No other launches for this product.