Nicheloom visits: 62,882 all-time · 8,415 in the last 7 days

The opportunity tracker for new startups.

Today: 1,441 launches · Niche of the day: digital marketing agency services · See what launched today →

ExANS

Lossless KV cache compression at 622 GB/s on H100

Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.

Want to reach these readers? Sponsor an issue.

Hi HN,We are the developers of OpenLake, an open source storage engine for KV cache offloading to remote disk and memory.Once we offloaded to local disk, we realized the bottleneck is the PCIe or NIC bandwidth. We wondered whether on GPU lossless compression is viable for fast reads and lower TTFT.BF16 is usually very hard to compress, (high entropy of sign/mantissa). What surprised us is that real world KV blocks are very different. The exponent byte has a very low entropy and barely populated. Instead of compressing the whole tensor, we compress only the exponent stream on the GPU.We see the following results: (H100, production KV snapshot):- 1.51× lossless compression - 622 GB/s median GPU decodeDecompression is ~10× faster than a 400 Gb/s NIC bandwidth delivering data losslessly without quality change.We've are open sourcing this as: ExANS which will be available through our vLLM and SGLang connectors on OpenLake v0.8 version. No changes are required in the inference engine.I'm curious how others are handling KV transfer today. Are you using KV compression or is bandwidth not a bottleneck yet?Thanks!GitHub: https://github.com/openlake-project/openlake Technical Blog: https://theopenlake.com/blog/exans-lossless-gpu-compression-...

HN · launched Aug 5, 2026 · 15 upvotes (top 30%) · 0 views

Niche: llm inference and deployment tools · 193 launches, 8 in the last 7 days ▲ +60% · Explore this niche →

Category and tags

  • Horizontal
  • Model & infra
  • Developer
  • AI-native
  • Commercial product

Visit website ↗ Source listing ↗

Could you build this?

No Lossless KV cache compression at 622 GB/s on NVIDIA H100 requires deep low-level CUDA/GPU systems engineering, custom ANS entropy coding implementations, and hardware-specific memory optimization.

What it would actually take: Building ExANS requires deep expertise in GPU architecture, high-performance CUDA/C++, and asymmetric numeral systems (ANS) compression algorithms. The developer must write custom GPU kernels that balance compression ratio against high warp-level parallelism and memory coalescing to saturate PCIe/memory bus bandwidths. Profiling with Nsight Compute and rigorous validation against LLM KV cache memory layouts are required.

Discussion

No comments on this launch.

Competitors

Other products that read as similar to this one : 362 launches clear the similarity bar, closest 8 shown.

Attention rank: #122 of 363 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).

Launched 280 days after the earliest competitor.

The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.