Nicheloom visits: 63,827 all-time · 8,415 in the last 7 days

The opportunity tracker for new startups.

Today: 2,659 launches · Niche of the day: browser-based pdf tools · See what launched today →

Smile-Serve

Inference Server for ML, ONNX, and LLM

Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.

Want to reach these readers? Sponsor an issue.

SMILE Serve is a production-ready inference server built on [Quarkus](https://quarkus.io/) that brings together three complementary inference capabilities on the JVM: - **Classic ML**: `/api/v1/models` for serialized SMILE models (`.sml`) - **ONNX Runtime**: `/api/v1/onnx` for any model in the ONNX open format (`.onnx`) - **LLM Chat**: `/api/v1/chat` for Llama 3 chat completions A React-based web UI is bundled and served from the same process.

HN · launched May 4, 2026 · 5 upvotes (top 89%) · 0 views

Niche: machine learning inference engines · 112 launches, 5 in the last 7 days ▼ -71% · Explore this niche →

Category and tags

  • Horizontal
  • Model & infra
  • Developer
  • AI-native
  • Hobby / open-source project

Visit website ↗ Source listing ↗

Could you build this?

Partial Building a Quarkus/Java REST wrapper around ONNX Runtime and ML models is largely straightforward, but managing low-level JVM off-heap memory, concurrent native runtime bindings, and high-throughput inference threading requires specialized backend systems expertise.

What it would actually take: The server uses Quarkus (or Netty) alongside Java Native Interface / Foreign Function & Memory (FFM) API wrappers for ONNX Runtime and SMILE bytecode execution. The difficult aspects are memory management for native tensor allocations to prevent JVM memory leaks, batching mechanisms, and fine-tuning thread pool isolation between CPU/GPU execution providers under high concurrency.

Discussion

No comments on this launch.

Competitors

Other products that read as similar to this one : 98 launches clear the similarity bar, closest 8 shown.

Attention rank: #92 of 99 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).

Launched 181 days after the earliest competitor.

The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.