Skip to Main Content

galactico

Built at Meta Global AI Developer Hackathon · Oct 26, 2026 · Remote

galactico — Demo video

Lumen is a live, interactive layer over reality for learning and doing physical tasks. Point your phone at what is in front of you and talk. Lumen highlights exactly what to touch, guides you step by step, catches mistakes and hazards, diagnoses why something is not working, and verifies that you got it right. Most physical skills are learned best beside someone who already knows: a teacher beside a circuit board, a cook beside a cutting board, or a technician beside a broken machine. Videos and manuals cannot see your workspace, your hands, or your mistake. As a result, physical errors are often silent, expensive, or dangerous — a fried ESP32 board, a ruined meal, an unnecessary TV replacement, or a cut finger. Lumen solves this by turning expert guidance into light projected onto the real world. It perceives the user's actual objects and hands, draws a pixel-accurate glowing outline on the exact pin, wire, ingredient, screw, or tool to use next, and speaks hands-free instructions. After the user acts, Lumen verifies the result using vision, deterministic rules, and real hardware signals. It can say "correct", "not yet", or honestly, "I am uncertain — show me closer." One reusable engine powers every domain through pluggable Skill Packs — single files that define objects, procedures, checks, hazards, faults, and vocabulary. The hero pack coaches users to wire an RFID reader, display, LED, and button to an ESP32-S3, verified by the board's own telemetry. The same app, with no code change, coaches knife skills in the kitchen, identifies and interacts with anything the user points at, and turns picture-book characters into talking 3D toys on Android. Safety is built in: serious hazards freeze progress in under 300ms using rules and telemetry, without waiting for a model. When something fails, users can say "it still reads nothing" and Lumen ranks likely causes, tests them live, and highlights the physical suspect. Experts can also film a task once and have Lumen turn that demonstration into a new coachable Skill Pack. Lumen makes scarce expertise available to learners, makers, cooks, repairers, students, and technicians — anyone doing a physical task without an expert beside them.

Team