FaceTime-style calls with an AI Companion (Live2D and long-term memory)
Details
- External ID
- 46759627
- Source
- HN
- Company
- —
- Product
- FaceTime-style calls with an AI Companion (Live2D and long-term memory)
- Website domain
- thebeni.ai
- Launched
- Jan. 25, 2026
- Cohort
- —
- Upvotes
- 34
- Upvotes percentile
- 0.733201581027668
- Tags
- —
- Fetched at
- Sept. 7, 2026, 9:25 p.m.
- Updated at
- Sept. 7, 2026, 9:25 p.m.
Description
Hi HN, I built Beni (https://thebeni.ai ), a web app for real-time video calls with an AI companion.The idea started as a pretty simple question: text chatbots are everywhere, but they rarely feel present. I wanted something closer to a call, where the character actually reacts in real time (voice, timing, expressions), not just “type, wait, reply”.Beni is basically:A Live2D avatar that animates during the call (expressions + motion driven by the conversation)Real-time voice conversation (streaming response, not “wait 10 seconds then speak”)Long-term memory so the character can keep context across sessionsThe hardest part wasn’t generating text, it was making the whole loop feel synchronized: mic input, model response, TTS audio, and Live2D animation all need to line up or it feels broken immediately. I ended up spending more time on state management, latency and buffering than on prompts.Some implementation details (happy to share more if anyone’s curious):Browser-based real-time calling, with audio streaming and client-side playback controlLive2D rendering on the front end, with animation hooks tied to speech / stateA memory layer that stores lightweight user facts/preferences and conversation summaries to keep continuityCurrent limitation: sign-in is required today (to persist memory and prevent abuse). I’m adding a guest mode soon for faster try-out and working on mobile view now.What I’d love feedback on:Does the “real-time call” loop feel responsive enough, or still too laggy?Any ideas for better lip sync / expression timing on 2D/3D avatars in the browser?Thanks, and I’ll be around in the comments.
Enrichment
- Theme
- task-specific ai agents and assistants
- Vertical
- Horizontal
- Function
- Agent / copilot
- Audience
- B2C
- AI stance
- AI-native
- Project type
- Commercial product
- Normalized one-liner
- video calling with ai companion
- Manually corrected
- False
Could you build this?
Partial While wrapping real-time voice and LLM APIs with a Live2D web viewer is straightforward, low-latency synchrony between conversational audio, real-time Live2D facial rigging/lip-sync, and semantic long-term memory retrieval is tricky to optimize.
What it would actually take: The architecture combines a WebRTC or WebSocket audio streaming server, an ultra-low-latency speech-to-speech/STT-LLM-TTS pipeline, and an embedding-based vector memory store. The hardest part is driving Live2D parameters (visemes, head rotation, eye tracking) synchronously with streaming audio packets while maintaining sub-second voice latency. Implementing this requires solid WebRTC audio pipeline engineering and custom Live2D/lip-sync animation mapping expertise.
Discussion
20 comments analyzed.
Competitors mentioned: ElevenLabs (TTS provider), ChatGPT/Gemini (baseline LLM), Live2D (animation parameters), Rhubarb (lip sync tool)
Concerns raised: Parasocial relationship dynamics with AI, Latency and real-time performance issues, Uncanny moments in interaction, Ethical concerns about therapy replacement, Tension between consistency and personalization at scale
Feature requests: Better lip sync using Rhubarb or viseme extraction, LLM diversification beyond ChatGPT/Gemini baseline, Improved turn-taking and conversational flow, Real-time streaming with lower latency
Competitors
Other products that read as similar to this one — 237 launches clear the similarity bar, closest 8 shown.
Attention rank: #86 of 238 (itself plus its competitors, highest first — normalized so YC and Product Hunt are compared fairly).
Launched 87 days after the earliest competitor.
- Beni AI · hn · 2026-01-15 · 9 upvotes · similarity 0.64
- I built an AI conversation partner to practice speaking languages · hn · 2026-01-30 · 65 upvotes · similarity 0.55
- Real-time avatars that change emotions as you talk · hn · 2026-07-14 · 12 upvotes · similarity 0.52
- Multimodal perception system for real-time conversation · hn · 2026-02-10 · 54 upvotes · similarity 0.49
- LemonSlice · hn · 2026-01-27 · 133 upvotes · similarity 0.48
- Realtime TTS-2 · ph · 2026-05-06 · 152 upvotes · similarity 0.47
- I built a voice AI that responds like a real woman · hn · 2026-03-25 · 6 upvotes · similarity 0.46
- Created an AI for myself to achieve goals, it might help you guys too · hn · 2026-01-19 · 6 upvotes · similarity 0.45
Other launches for this product
- No other launches for this product.
Same idea, different domain
Nobody's really built a agent / copilot tool for Agriculture yet.