# BITS Pilani

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Aviral saxena](https://cerebralvalley.ai/u/Ninja11)
- **GitHub:** https://github.com/AviralDeshratna11/MetaAIHack
- **Demo video:** https://youtu.be/FgyGmL5OSXM
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3486

Video2Robot turns ordinary human manipulation videos into traceable candidate demonstrations for robot learning, addressing the cost and effort of collecting structured robot training data.

The pipeline combines Meta’s Muse Spark for semantic action understanding, SAM for segmentation and tracking, and VGGT with SAM 3D Body for scene and body reconstruction. It organizes these outputs into an interaction trajectory containing object-relative motion, contact hypotheses and explicit uncertainty.

A robot compiler adapts supported motion to Franka robots, generates joint and gripper commands, and checks trajectories through inverse kinematics and MuJoCo physics simulation. Results can be exported as LeRobot datasets and RLDS episodes, including simulated observations, actions and configured task rewards.

An inspection interface connects source video, tracking evidence, reconstruction, simulation and downloadable datasets. Every result retains provenance, assumptions and quality reports so researchers can assess its suitability before training.

The current prototype demonstrates a short, review-gated motion adaptation with verified dataset exports. Complete source-task replication and physical robot deployment remain unvalidated.

## More from Meta Global AI Developer Hackathon

- [galactico](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3483)
- [BridgeVoice](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3484)
- [Niko Packbier](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3485)
- [mahir](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3487)
- [Victorious](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3488)
- [Gumorose](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3489)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3486. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt