Clippy
screen-aware voice AI in the browser
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
A friend and I built a browser prototype that answers questions about whatever’s on your screen using getDisplayMedia, client-side wake-word detection, and server-side multimodal inference.Hard parts:– Getting the model to point to specific UI elements– Keeping it coherent across multi-step workflows (“Help me create a sword in Tinkercad”)– Preventing the infinite mirror effect and confusion between window vs full-screen sharing– Keeping voice → screenshot → inference → voice latency low enough to feel conversationalWe packaged it as “Clippy” for fun, but the real experiment is letting a model tool-call fresh screenshots to help it gather more context.One practical use case is remote tech support — I'm sending this to my mom next time she calls instead of screen sharing.Curious what breaks.
HN · launched Mar 18, 2026 · 5 upvotes (top 89%) · 0 views
Niche: voice productivity assistants · 126 launches, 7 in the last 7 days ▼ -22% · Explore this niche →
Category and tags
- Horizontal
- Agent / copilot
- B2C
- AI-native
- Commercial product
Visit website ↗ Source listing ↗
More like this: all 126 launches in voice productivity assistants →
Could you build this?
Partial While standard Web APIs handle media capture, grounding real-time multimodal model responses to precise dynamic UI screen coordinates with low latency across complex user interactions exceeds trivial vibe-coding.
What it would actually take: The stack uses WebRTC/LiveKit for real-time video/audio streaming, Porcupine or WebAssembly for client-side wake-word detection, and vision models (like Gemini Live or GPT-4o Realtime) for inference. The hardest problems are spatial grounding (mapping screen bounding boxes to DOM elements or mouse coordinate overlays in real-time) and maintaining reliable conversational state across multi-step browser interactions.
Discussion
No comments on this launch.
Competitors
Other products that read as similar to this one : 231 launches clear the similarity bar, closest 8 shown.
Attention rank: #220 of 232 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 132 days after the earliest competitor.
- FaceTime-style calls with an AI Companion (Live2D and long-term memory) · hn · 2026-01-25 · 34 upvotes · similarity 0.44
- Voice-tracked teleprompter using on-device ASR in the browser · hn · 2026-03-15 · 6 upvotes · similarity 0.43
- Assist · ph · 2026-09-07 · 128 upvotes · similarity 0.43
- Cliptate · ph · 2026-10-12 · 1 upvotes · similarity 0.42
- CLIPARY · ph · 2026-10-26 · 1 upvotes · similarity 0.41
- voice-workshop · github · 2026-11-08 · 36 upvotes · similarity 0.41
- Multimodal perception system for real-time conversation · hn · 2026-02-10 · 54 upvotes · similarity 0.40
- Jarvis AI · ph · 2026-10-29 · 1 upvotes · similarity 0.40
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.