Simon
Built at Meta Global AI Developer Hackathon · Oct 26, 2026 · Remote

Simon says, but also Simon shows. Simon is a good boy. Simon can take any movement video, analyze it, turn it into a full 3d assessment, then share that movement with anyone else to follow. Simon will break down any video into keyframes, analyze those distinct keyframes, convert them into 3d mappings, generate the transitional movements between each keyframe, and create a fully quantifiable video to score against. Simon does this by utilizing many frontier AI models to turn any simple video into a full motion capture analysis. Did you follow the instructions and routine or did you stray? By how much? Is it a habit of yours? Simon uses the latest meta frontier AI models for tracking your behaviors over time using historic context, AI models to render 3d bodies, AI to generate a 3d model for you, and much more. People learn to move by imitating, a dance routine on video, a certain exercise in the gym or a way to stretch from a physio. But you move, and the only way to get feedback is to have an expert in the room, which isn’t always feasible. Simon is here to bridge that gap. You can get the expertise knowledge from all top frontier Meta models to act as your judge. No need for motion capture suits or wearables. You only need your phone camera. Have a new dance routine or a folk dance, no need to reprogram it in the app. Just one take of it and Simon turns it into a 3D reference that anyone can follow. Users capture their own attempts using any standard phone or laptop camera, regardless of the angle. Simon instantly processes their movement in 3D, matches the timeline to the target video, and evaluates execution frame by frame across every joint. It then assigns a grade from F to S, generates a side-by-side 3D breakdown, and offers clear, actionable feedback such as "left arm drops early at 4.2 s" with direct links to exact timestamps. Simon also turns this feedback into actionable items, uses the follower’s model and reframes it in the position matching the video, “How it should be done view”. It also answers questions about the take using the original creator’s spoken rules. Why it matters: - Any movement, including ones no app knows. - One creator, a whole crowd. SAM 3.1 tracks every person in a group video - Challenges between friends. A fair answer to "who did it best?" - No equipment, any language. A phone instead of a motion-capture studio. - Your own character. Replay a take as a character made from one picture with SAM 3D Objects and MHR simon.live has been live with real users since November 2, and a 10-second take is scored in about 80 seconds for roughly 2¢ of GPU time. That cost makes a standard limited free tier, premium tier, and paid trainer and studio plans viable. Full design, measurements and roadmap: https://simon.live/whitepaper.html