Spala
Built at Meta Global AI Developer Hackathon · Oct 26, 2026 · Remote

StepWitness is an agent that watches a video of someone building something and checks it against the step list. It reports steps done out of order or skipped: the second, the step, and a frame with the part boxed. Then it re-checks its own flag blind. It runs only on Meta's Muse Spark 1.3 and SAM 3.1. PROBLEM: Mistakes in an assembly are silent. A part goes on before the one it depends on, the build looks almost right, and the error shows up later as rework, a return or a failed inspection. Training a new line worker, checking a technician's install and helping someone with flat-pack all end the same way: a person scrubbing through video, asking where it went wrong. WHAT IT DOES: Takes a step list with prerequisites (plain text or a picture-only manual that Muse Spark reads into steps). Watches the build in overlapping windows and keeps an evidence record of what was attached, when, and in which frames. A deterministic order check compares the evidence with the prerequisites and raises one primary flag per build: the second, the step and a one-line reason (wrong order and skipped steps). A blind re-check looks again at still frames plus a SAM hand zoom, without being told what was flagged, and labels the flag verified, disputed or unverified; it never hides the flag. SAM 3.1 boxes the part in the builder's hand at the flagged second. Every run records each turn, tool call, observation and cost under a hard spend cap and can be replayed in the console. Other agents can hire it over A2A or plain REST. VISION: If a video or written guide shows how to do something, StepWitness should watch you do it and tell you how you did: workouts, furniture and electronics assembly, repairs, cooking, lab prep, crafts, on-the-job training. ROADMAP (planned, not built): any guide (YouTube tutorial, expert demo, SOP) as the reference; live camera feedback; hands-free on Ray-Ban Meta glasses; form and technique checks; proof and progress records for training sign-off. HONEST LIMITS: 30 runs on 24 clips cut from openly licensed videos with planted mistakes: swapped steps caught on 5 of 7 clips, skipped steps 3 of 6, false alarm on 3 of 8 clean clips. The blind re-check is not reliable yet. Median run cost $0.21.