athlete2humanoid

Training a humanoid reinforcement learning policy takes too much time, expertise, and equipment. Every new skill needs a motion capture session, a retargeting pass, a hand-written reward, and an engineer to tune it, so most of the people who could teach robots something useful never get to. That bottleneck slows physical AI research and the commercialization of humanoid robotics. The day anyone can train a humanoid robot is the day physical AI is truly part of society, and athlete2humanoid is one big step toward it. athlete2humanoid turns a single phone video of a person doing an action into a training environment, a dataset, and a trained Unitree G1 policy. There is no mocap studio, no motion library, and no hand-written reward. Anybody with a phone can train a humanoid robot. Perception. SAM 3.1 segments and tracks the person from a text prompt, so the pipeline works on a crowded frame or a shaky phone clip and the mask crops the frame for everything downstream. SAM 3D Body lifts the masked person into a metric 3D skeleton of 17 joints. We chose it over 2D-plus-depth pose estimators because its bone lengths are stable across frames, which is what retargeting needs: on our phone clips it cut bone-length variation 3 to 6x and halved knee error. Retargeting. The skeleton is rebuilt on the G1’s 29 joints by proportion-aware inverse kinematics in mink. Rather than copying joint angles or scaling the person uniformly, it solves for the robot’s own bone lengths, so a person with long legs and a robot with short ones still land their feet in the same place. This cut knee error from 11.9 to 3.0 degrees and stance-foot sliding from 29 to 0.4 cm/s. Task compilation. The motion is turned into an ordered sequence of key moments with the Ramer-Douglas-Peucker algorithm. RDP simplifies a trajectory by keeping only the points that deviate most from a straight line between their neighbors, which makes it a significance detector: the moments it keeps are where the motion changes direction, such as the bottom of a squat or the apex of a jump, and the moments it drops are transitions. The number of key moments falls out of the motion itself instead of being fixed in advance. Muse Spark then watches the clip and names each moment (“sink into right lunge”, “arms overhead legs apart”), so the task spec is human-readable and editable. Dataset. One clip cannot teach a robust policy, so the pipeline generates mirrored, re-timed, and rescaled variants of the move and checks each against its own spec and a physics filter. A policy trained only on the original falls on every mirrored variant; one trained on the set never falls on a held-out one. Training. 4,096 simulated G1s train in parallel with PPO in mjlab on MuJoCo Warp, so a skill is minutes of training rather than days. V-JEPA 2 is Meta’s video world model: it is trained to predict video in latent space, so its embedding encodes how a body moves rather than what it looks like. That makes it useful in three places. Its embedding of past frames is an input the policy sees, which raised success from 44% to 75% and removed falls across five clips. A style term rewards rollouts whose embedding matches the video. And it judges likeness after training. We also tested it as a forward predictor and as the only reward; neither held up, and we report both. The agentic loop. Between training rounds, Muse Spark reads the evaluation, watches the rollout video, and calls bounded tools to change reward weights and termination settings, giving a reason for each change. The next round trains from the last checkpoint beside an untouched control with the same budget. Over three rounds it raised V-JEPA likeness from 0.388 to 0.410 against the control. This was emergent: we gave it tools and metrics, not a strategy, and it chose to raise the tracking and style weights while leaving the stage bonuses alone, and its own end-of-run summary matched the numbers. As a blind critic it also caught a bug our checker missed, noticing the robots “never really jump” when they were tuck-hopping with the pelvis never leaving standing height. Results. A new move adapts in about 18 minutes on one GPU, warm-started from a generalist. Policies hold on held-out variants they never trained on and recover from pushes up to 1 m/s. Everything runs live at https://athlete2humanoid.pages.dev. Upload a clip, watch the environment build stage by stage, train on a GPU, and shove the robot. Or download the environment, task spec, variants, reward config, and loader and train it yourself with mjlab.