# LogicCraft

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Woosik Choi](https://cerebralvalley.ai/u/cws)
- **GitHub:** https://github.com/Woosiq92/stepcards
- **Demo video:** https://youtu.be/WUuhKDdM5CM
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1876

For job training of people with developmental disabilities: a teacher films a task once, and StepCards turns it into checked picture step cards in minutes. Special-education teachers make these cards by hand today, and one set takes hours. While the learner follows along, Muse Spark watches the camera and shows the next card when a step is done, so nobody has to stand beside them turning cards.
- Try it with no camera: open https://stepcards-production.up.railway.app/?k=9ff0029689f67382 (signs you in as a judge) and press "See it work · no camera". The napkin sample plays in place of a camera while Muse Spark turns the cards.
- Demo video (3 min): https://youtu.be/WUuhKDdM5CM
- So far: a 2-minute-20-second napkin video became 12 checked cards in 59 seconds. Replaying demonstration videos as the camera, all eight runs reached the last card on their own. We have not yet tested it with real learners.

The problem
Today, special-education teachers film a task, pause, take a screenshot, write and print, card by card. The library already has cards for 3 of the 11 jobs that Korea's 2026 disability jobs guide lists for people with severe disabilities. The same gap shows up wherever work is learned by watching: a first week at a cafe, a family recipe, a dance from a Reel. A single demonstration goes by too fast.

What it does
1. Make: drop in a video filmed at eye level, or press "Record with camera" to film it in the browser. Muse Voice Transcribe writes down what is said. Muse Spark splits the video into steps of one action each, with timestamps and a short sentence in Korean and English. Our code checks each card for overlapping timestamps, long sentences and two actions in one card, and sends any card that fails back to the model with the reason. SAM 3.1 picks a photo for each card and outlines the object in the hand. Muse Spark also says whether the video is real footage or a drawn manual. In a drawn set, cards turn by hand, because a drawing can't be matched with real objects on camera. In one recorded run, a 2-minute-20-second video of folding a napkin became 12 cards in 59 seconds, for about 8 cents of Muse Spark and transcription, plus about 6 cents of SAM 3.1 calls by our estimate. The run shown in the demo video made 12 cards from the same napkin video in 75 seconds, at the same cost.
2. Review: the maker can edit any card, and every edit is checked again. Yellow notes flag numbers that nobody said in the video, and left and right, which are easy to swap. Each card can use a photo, a clip from the video, a Muse Image drawing or a 3D fold rebuilt from the demonstration.
3. Ask: the maker can also ask in plain words, for example "simpler words for a learner who reads little." Muse Spark gets our rule checker as a tool. It runs the checker on its own draft and fixes what it reports before it answers. Our code then checks the final answer again. Nothing changes until the maker taps Apply. In a run on the napkin cards (Nov 3), it proposed new sentences for 7 of 12 cards for about 4 cents, and the answer took 51 seconds. In the run shown in the demo video, it proposed new sentences for 10 of 12 cards and ran the checker twice. That took 60 seconds and cost about 5 cents.
4. Follow: the camera view stays large next to the card. Muse Spark compares the last two camera frames with photos of the finished steps in the demonstration and decides when to move on. When Coach starts, it checks that the camera sees the work area the way the demonstration does, and lays a faint demonstration photo over the camera view to line up with. This doesn't hold back the decisions. For sets filmed facing the camera (dance, sign language), an on-device pose check says right away which body parts are out of view. When motion stops, the app asks Muse Spark once more right away instead of waiting. Before a live camera frame goes to the Meta API, the browser finds faces and pixelates them, so Muse Spark sees the work and the hands, not who is doing it. If the face model cannot load, no frame is sent and cards move by hand. Sets filmed facing the camera (sign language, dance) are not blurred, because the face is part of the movement. The demonstration video itself is not blurred when cards are made either. Help steps up from text to a picture, then to the sentence read aloud, then to a motion clip. Pressing "Ask for help" puts the card number on a live board. When the helper presses "On my way", the learner's screen and glasses show "Help is coming".
   Finishing a run once does not show that the learner can do the task. So the app saves each run on the device: for every card, how much help the learner needed and whether the camera saw the step finished. The end screen compares this with the last run. A card the camera saw finished last time starts with a still photo instead of the motion clip, and the clip comes back if the learner gets stuck. After one full run, a third stage hides the cards while the camera keeps judging. Three full runs in a row with no card opened, and every step seen finished by the camera, mark the set "Can do it alone".
5. Glasses: a 600x600 view for Ray-Ban Display shows the same card and takes Back, Help and Next with the D-pad. The glasses show the cards. A phone or laptop camera does the watching, because the glasses web app has no camera access. Opened on its own, the glasses page lists the card sets and offers "See it work". We tested it in Meta's Ray-Ban Display simulator, not on a device.
6. Local option: card making also runs on an Apple Silicon Mac with 48 GB of memory and the open-weight Muse Glimmer, so the video never leaves the Mac. It skips speech transcription and SAM outlines there, since those are Meta API calls. There, a napkin video became 11 checked cards in 172 seconds, with no API cost.

Measured so far
- Replaying the demonstration video as the camera: in eight runs on the napkin, latte and towel videos, every run reached the last card on its own. The median move to the next card came 6.8 seconds after the step ended in the demonstration, and 5 of 96 moves came more than 2 seconds early. Before we switched to photo matching, the napkin run stopped at card 6 or 7 of 12, and the towel run at 8 of 15. This is the video the cards were made from, so it is an upper bound.
- Asking again when motion stops: in paired runs side by side at the same time, the median lag went from 9.6 to 8.2 seconds and the worst from 35 to 20 seconds. It cost about 30% more calls, and 2 moves came more than 2 seconds early (none did without it).
- Videos the cards were not made from: Muse Image redrew each step of the napkin and towel tasks with a different cloth, table, hands and camera angle, and we joined the frames into new videos. We changed the Coach logic using one of them (white paper napkin) and ran the other two only at the end. Each pair ran side by side at the same time, before and after the change:
  - white paper napkin (used for tuning): 6 and 5 of 12 cards → 9 and 9
  - navy napkin on a marble table: 6 and 7 of 12 → 9 and 11
  - green towel on a dark table: 11 and 11 of 12 → 12 and 11
  No move came more than 2 seconds early, and a run still cost about 15 cents.
- What changed: when the learner is stuck or ahead, Muse Spark sees every remaining step's result instead of the next two; it can answer "further along than any of these"; and for steps that look the same before and after, like flipping a napkin, it watches the last four frames for the movement.
- One idea did not help: labelling each reference photo with a text description of the folded shape (9, 9 and 9 cards with it, 9 and 12 without). We removed it.
- When Coach starts, Muse Spark checks that the camera sees the work area the way the demonstration does. A seated laptop camera for the napkin got "Point camera down at work table"; in an earlier version a seated signer got "Step back to show your upper body" (facing sets now use the pose check instead).
- The footage check sorted 3 videos correctly: an animated assembly manual as drawn, the napkin and dance videos as real.
- On 11-02, Meta API decision latency changed with the time of day: about 2.3 seconds in the morning, 4 to 13 seconds in the afternoon.

Limits
We have not tested Coach mode with real learners or with a second real person; the videos above are generated stills joined together, with no real hand movement between them. Signing is too fast for the judging loop: replaying the 5-minute sign video, which shows a new sign about every 5 seconds, Coach reached card 15 of 61 (12 without the movement check). Dance repeats the same moves, and Coach moved on early all 4 times. Face blur was measured on a synthetic webcam (1 face missed in 141 frames). We checked the glasses view in the simulator only, and have not tried iPad Safari or Chromebooks. Coach mode costs about 7 cents per minute of following, so learner devices without a key share a daily limit per card set. The run history stays in one browser, and the three-runs rule comes from common special-education practice; we have not checked it with learners.

Try it
- Samples are read-only. To try editing and Ask, press "Edit" on a sample; it makes a copy that is yours. Copies and new card sets belong to the browser that made them, so judges sharing the key don't see each other's work. The judge key has a shared daily limit.
- Code: https://github.com/Woosiq92/stepcards
- Demo video: https://youtu.be/WUuhKDdM5CM

Built during the hackathon
We wrote all the code during the hackathon, starting Oct 28, 2026. The commit history shows it. The sample videos are CC BY and are credited in the app and README.

## More from Meta Global AI Developer Hackathon

- [SafeRoute](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1873)
- [Ade Yusuf Rahardian](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1874)
- [PushPilot — From Sight to Action](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1875)
- [Nunchi](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1877)
- [Llama Guard Sentinel](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1878)
- [SparqCode](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1879)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1876. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt