# Scout

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Tayyab Khan](https://cerebralvalley.ai/u/tayyab415)
- **GitHub:** https://github.com/tayyab415/scout
- **Demo video:** https://scout-demo.d157s5yat0cu2x.amplifyapp.com/demo.mp4
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4478

Scout is a full Meta stack for robot simulation: a Meta Motivo humanoid body, SAM 3.1 eyes and a plug-in brain (Muse Spark 1.3), running together in a physics-simulated home, plus a V-JEPA 2 world model trained on the robot's own eyes. It tests one thing every home robot will need: knowing when its memory has gone stale, and walking back to check.

The robot explores a house it has never seen and remembers what it saw and where. Then we split the world in two. In one copy nothing changes; in the other we quietly move an object. Both copies get the same question: "Is the sofa still where you saw it?" Memory says yes in both worlds, so it is wrong in one. A robot that goes back and looks gets both right.

How we got here. We started deep in JEPA: we wanted a robot that learns a world model from its own eyes. We read the V-JEPA 2, Meta Motivo and SAM 3 papers and built a world model on a frozen V-JEPA 2 encoder, trained on 104,730 frames from the humanoid's own head camera. It learned what its actions do to its view in houses it had never seen, but it could only walk to a goal photo 3 times in 36. While running it, we hit the real gap: a robot's memory of a home goes out of date, and nothing measured whether it notices. Meta's OpenEQA tests answering from memory, but not the case where the world changed after the memory was formed. That became Scout.

How it works:
- Body: Meta Motivo, a behavioural foundation model, controls the full humanoid at 30 Hz in MuJoCo. We blend its skill vectors into walking and turning to look, and it gets itself up after falls: 35 falls, 35 recoveries in the final run.
- Eyes and memory: SAM 3.1 segments 16 object types once per second on a 512x512 RGB-D head camera. Each mask is lifted into 3D with depth and stored with the frame it came from, so every answer can cite its evidence. The robot builds its own floor map; it never sees the layout.
- Two worlds: the physics state is saved and restored exactly (replay differs by 5.8e-12), so both worlds start identical.
- Grading: a hidden evaluator knows what was really in view, frame by frame. Every answer and every cited frame is checked against it.

The neural network. Every episode becomes training data: the robot's memory, paired with what was really there. We trained a 233-parameter neural network on it (27 inputs, 8 hidden units, 1 output) in seconds on a laptop CPU. It learned which of SAM 3.1's sightings to doubt. Put in charge of the robot in eight homes it had never seen, it scored 106/112 on the memory quiz, first of five brains, ahead of a frontier LLM (92).

Results across 32 final episodes: six clean verification pairs where looking beat memory in both worlds; answering from memory was right 0 times when the object had moved.

## More from Meta Global AI Developer Hackathon

- [Will Wild](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4475)
- [FPGA Sheperd](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4476)
- [SAMgo](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4477)
- [Skrpt_Kpr404](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4479)
- [sstmade](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4480)
- [SpecCheck](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4481)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4478. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt