# Video2Sim

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Tom Korbmacher](https://cerebralvalley.ai/u/toko07)
- **GitHub:** https://github.com/t0mezz/video2sim/tree/dry
- **Demo video:** https://youtu.be/377SPZHlCyI
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4662

Video2Sim turns one handheld smartphone video of a tabletop into a scene a physics simulator can run: a metric-scale USD file plus one URDF per object. Each object has a closed mesh, collision geometry, mass, inertia and friction ranges.
The motivation was to demonstrate, how in the future, anybody will be able to easily train robotics for their  own specific setup.

For judging:

origin/dry provides a possibility to dry run the pipeline and view the result, if you don't want to setup the SAM3D inference.
The entire project as currently in main and dry was created during the span of the hackathon, except for the dataset, which I captured in advance.

check `https://vts.kortec.me/judges`, I may later add a method so you can run the SAM3D inference over my setup.

EDIT: `video2sim-124190` to get the key on the form

## More from Meta Global AI Developer Hackathon

- [CreativeBuilders](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4659)
- [OdeMuse](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4660)
- [MAV | Mobile Agentic Vision](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4661)
- [Movement Tracker+](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4663)
- [Jayly](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4664)
- [SLVRLeaf Digital](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4665)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4662. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt