I built a new software primitive. It replaces AI screenshot agents
Get picks like this daily. Today's launch numbers, the top launches, AI and tech news, and a daily opportunity pick, straight to your inbox.
Want to reach these readers? Sponsor an issue.
I built a Rust binary (~700KB, two dependencies) that snaps onto any Windows application, continuously reads its UI Automation accessibility tree into a SQLite database, and lets external processes control the app via SQL INSERT.The core insight: every GUI application already describes itself as structured text through the accessibility layer (built for screen readers since 1997). Every major AI lab is taking screenshots of this and running vision models on it. DirectShell just reads the text directly.What it does: - Reads every button, field, menu item into a queryable SQLite DB (refreshed every 500ms) - Generates multiple output formats: full DB, interactive elements list, LLM-optimized snapshots (50-200 tokens vs 1,200-5,000 for a screenshot) - Controls apps via 5 action types: click by element name, set text via UIA ValuePattern, type character-by-character, send key combos, scroll - Includes an MCP server so Claude/GPT can use it directlyDay 1 demo: filled 360 Google Sheets cells in 90 seconds, read and replied to a Claude.ai conversation cross-app, wrote to Notepad instantly. No screenshots, no vision model, no coordinate guessing.Limitations (honest): built in 8.5 hours, single-app scope, Chromium apps need a 4-phase activation hack to expose their tree, accessibility quality varies by app. AGPL-3.0.Demo: https://youtu.be/nvZobyt0KBg Full technical paper: in the repo under Dokumentation/ and on https://dev.to/tlrag/i-built-a-new-software-primitive-in-85-...
HN · launched Feb 17, 2026 · 5 upvotes (top 89%) · 0 views
Niche: computer and browser automation agents · 127 launches, 0 in the last 7 days ▼ -100% · Explore this niche →
Category and tags
- Horizontal
- Dev tools
- Developer
- AI-native
- Commercial product
Visit website ↗ Source listing ↗
More like this: all 127 launches in computer and browser automation agents →
Could you build this?
Partial Writing a bridge in Rust that inspects the Windows UI Automation tree and stores nodes in SQLite is a well-defined systems task, but handling the full breadth of Win32/UIA edge cases, IPC, and reliable bi-directional control requires low-level OS expertise.
What it would actually take: The binary uses the Windows UI Automation COM API (via the `windows-rs` crate) to listen to window lifecycle and automation events, serializing UI elements into SQLite tables with triggers or polling loops for incoming SQL INSERT commands to dispatch native synthetic actions. The difficult aspect is handling flaky accessibility trees, race conditions across varied frameworks (WPF, WinForms, WinUI, Electron), cross-session privileges, and thread synchronization. It requires solid systems programming skills and deep familiarity with Windows internals and the Win32/COM accessibility subsystem.
Discussion
No comments on this launch.
Competitors
Other products that read as similar to this one : 65 launches clear the similarity bar, closest 8 shown.
Attention rank: #62 of 66 (itself plus its competitors, highest first, normalized so YC and Product Hunt are compared fairly).
Launched 6 days after the earliest competitor.
- Agent-desktop · hn · 2026-05-02 · 99 upvotes · similarity 0.51
- I spent 3 months making desktop automation stop lying to AI agents · hn · 2026-08-15 · 6 upvotes · similarity 0.41
- Monospace from Directus · ph · 2026-10-01 · 394 upvotes · similarity 0.39
- Markdown is 40Ă— more token-efficient with SQLite · hn · 2026-10-27 · 10 upvotes · similarity 0.38
- Visual Workspace for Agents Based on Unix · hn · 2026-06-25 · 10 upvotes · similarity 0.38
- What should the GUI for AI agents look like? · hn · 2026-07-31 · 139 upvotes · similarity 0.37
- Direct SQL Access to Your QuickBooks Data · hn · 2026-09-14 · 5 upvotes · similarity 0.36
- I built a native macOS Markdown viewer 100% with AI coding agents · hn · 2026-05-19 · 6 upvotes · similarity 0.36
The niche, labels, vibe code rating and competitor comparison on this page are analysis by Nicheloom. If you use them, please credit nicheloom.com. How you may use our data.