# GetUp

- **Event:** [Gemini 4 NYC Hackathon](https://cerebralvalley.ai/e/gemini-4-nyc)
- **When:** Sat, Nov 21 at 9:00 AM – 10:00 PM (EST)
- **Where:** New York, NY
- **Team:** [James Burrell](https://cerebralvalley.ai/u/JBurrell), [Isaac Velez](https://cerebralvalley.ai/u/iva585)
- **GitHub:** https://github.com/JBurrell999/GetUp
- **Demo video:** https://www.youtube.com/watch?v=VxAyHNnZPEc
- **Gallery:** https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/36

GetUp: turn any phone video into humanoid robot motion

Teaching a humanoid a new task usually takes motion-capture suits, calibrated cameras or hours of teleoperation. GetUp needs one casual phone video, filmed from any angle with no calibration.

Film someone picking up a bottle, moving a chair or walking across a room, and a simulated Unitree G1 with three-fingered Dex3 hands does the same thing in MuJoCo. It's rendered from the same camera viewpoint and in the same room.

Gemini does the understanding in two closed loops:

Environment. Gemini watches the video and rebuilds the scene: tables and counters, the object's type, size and starting pose, the camera's position and field of view, and the person's starting posture. We render a still from the estimated camera and put it next to the real first frame. Gemini compares the two and corrects the scene until it matches, and then the scene is locked.
Motion. Gemini writes timed keyframes for the pelvis, trunk, legs and arms, plus when each hand grasps and releases and where the object ends up. Our replay smooths the keyframes, keeps the feet on the floor, uses 7-DoF arm IK to put each palm on the object, curls the fingers until they touch it, and checks for collisions ("right hand inside table 4 cm, 4.2–5.7 s"). Gemini then watches a time-synced side-by-side video of the human and the robot, reads the telemetry, scores how closely they match and revises the motion. If a round gets worse, the loop goes back to the best version so far.
On our benchmark of raw phone videos, the best runs reached 8.5/10 on both the bottle and chair tasks, with grasp and placement errors under 2 cm.

A web capture site walks people through a library of tasks, from calibration poses to walking, jumping and handling objects. A worker runs each upload in the room the person filmed in (hackathon room, kitchen or warehouse) and posts back the robot video, the side-by-side and the score.

Next: physics-based RL policies with domain randomization, so the motions can move from simulation to a real robot.

## More from Gemini 4 NYC Hackathon

- [Cornell Movers](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/33)
- [SightSense](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/34)
- [Uno ReVoice](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/35)
- [Work Less Video Editor](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/37)
- [Blindspot](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/38)
- [FightRoom UFC](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/39)

---

Markdown version of https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/36. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt
