AugentFlow

**Augent Companion Vision — a desktop companion that can finally see, and never clicks before you approve.** Computer-use agents today guess pixel coordinates from raw screenshots, so users cannot verify what will be clicked until it happens. Companion Vision grounds every action in segmentation instead: SAM 3.1 segments the current screen into concrete, numbered elements, Muse Spark plans the next step against that element map, and the companion highlights the exact target before anything runs. Nothing consequential executes without the user: an approval gate pauses every risky step, a Stop button ends a run at any moment, and every run re-captures and verifies the result before continuing — see → segment → plan → ask → act → verify. **What was built during the hackathon (27 Oct – 3 Nov 2026) — this is the new work the demo shows:** - The full vision engine (`internal/vision/`): screen capture, SAM 3.1 segmentation, Muse Spark planning, approval gating, mouse/keyboard execution, verification loop, plus a goal compiler (L1) that turns a vague goal into numbered steps and asks one clarification question when ambiguous. A no-progress brake ends stalled runs, and a run that cannot reach its goal reports itself as stopped with the planner's reason — never as a fake success. - The vision UX in the companion widget: vision mode on the single composer, a vision room showing run status, approval cards (including answering an ambiguous plan and resubmitting), and a Stop button that ends a run cleanly with "Stopped by the user." - Live validation on real, no-API apps: multi-step form filling, a note written and saved in a native text editor, file organisation in a file manager, and a real task on the live Cloudflare dashboard — navigating to a tunnel by name and opening it — re-validated end-to-end after the final round of fixes. **What already existed before the hackathon (not claimed as new work):** AugentFlow itself — the local-first desktop AI orchestration app (Wails/Go + React + SQLite), its agent runtime, task queue, approval-gate concept, and companion widget shell. Companion Vision is a new feature built inside that existing product, in the Startups track. **Repository note (honest scope, matches the README):** the public repo contains the complete hackathon-built vision package, its bindings, and the companion UI — for jury review, not as a standalone build: the bindings attach to AugentFlow's host task store and config, which are not included, and there is no `go.mod`. Source is public to read under All Rights Reserved (no use/modify/distribute rights granted).