Translator

Film a job. See which robot can do it, before you buy one. Buying a robot today is a bet. A spec sheet won't tell you if it can do your job, so you often find out after you pay. Then you teach it from zero, and the next robot from zero again. Translator turns one short video of a hand doing a job into a skill card: the objects, the grip, the steps and a success test. The card says what the job means, not how an arm moves, so a small arm and a humanoid can read the same card. Nine robots read it. Each one checks the card against its own body. If it can't do the job, it says why: "needs two hands", "must walk", "too heavy", "too big for its gripper". The others try it in MuJoCo and find the objects in camera images. A try counts only if the object ends up where the card says. Then you can mess with it. Move the cube, knock it out of the gripper. The card holds the goal, not a path, so the robot looks again and goes back. Results: - A video becomes a card in 52-169 s for $0.07-0.49 of Meta API cost. - 11 skills on 9 robots, 10 random layouts each: 611 of 760 tries passed (80 %), in the 76 cells where a robot can try. - Messing with the cube on 6 robots, 420 tries: a plan made once passed 49. Going back to the card passed 372 with the live loop and 398 with a simple re-plan. The lemon is harder: 180 of 420. - Muse wrote adapters for 8 robots from their model files. 6 passed with 9/10 or 10/10. TALOS and Booster T1 got 1/10, so they were not added. - 7 robots each trained an ACT policy on 200 checked episodes. On 50 new layouts they scored 19/50 to 37/50, well behind planned motion. Try it: UR5e, Panda and xArm7 run in your browser with MuJoCo. Drag the cube, knock it out, or set a job and see what 9 robots answer. https://thenguyentrong.github.io/skill-translator/ What isn't done: I have no real robot, so all of this is simulation, and a pass means the robot's model passed in MuJoCo. The test videos are Pexels stock clips. Humanoids slide on a base, their legs don't step. The door and carry skills in the library are hand-written. I built the whole project solo during the event. The repo starts on 26 Oct, after kickoff, and the commits are public.