Skip to Main Content

ActuateOS turns a video of a person assembling something into a working 3D robotic setup and tests whether a robot can actually pull it off. Watching a human do a task on camera is the easy part. Figuring out how a machine can repeat it safely is hard a video won't tell you if a robotic arm can reach a part, if pieces will collide, or if a step requires delicate human touch. ActuateOS bridges that gap automatically: It watches and learns: Using Gemini 4 Hydrogen, it breaks an assembly video down into clear steps, noting what happened and what information is still missing. It designs the workspace: Using Gemini 4 Carbon, it lays out a 3D robotic workcell in OpenIndustries and builds a step-by-step task plan. It puts the plan to the test: It runs the steps inside a real physics simulator (MuJoCo). Right now, our virtual robot arm handles standard pick-and-place moves and if a step is too complex (like tight screwing or snapping parts together), ActuateOS flags it honestly instead of faking success.

Team

ysonghkoushikvasaisayac+1

More from Gemini 4 NYC Hackathon

We design an agentic system to automatically create + manage intelligent AI opponents/NPCs in video games. The suite of agents analyzes user gameplay…

Adaptive NPCs project preview

Manually routing PCBs is time-consuming and cost-inefficient. Current best algorithms (freerouting, PCBRouter) score very low on PCBWorld's hardest PCB…

routr project preview

Heliocam is a camera app where Gemini controls the camera. It watches what you're pointing at, sets the ISO, shutter, focus, white balance and zoom for you…

heliocam project preview

Repairs and assembly fail because manuals are lost, vague, missing, or nonexistent, so people break things or throw them out. KLIK turns a photo of any object…

Klik project preview

Iron Pulse turns your laptop into a 3D boxing ring. Fight an adaptive AI rival in real time using just your webcam. Throw real punches and lean to dodge…

Iron Pulse Arena project preview

Text your raw footage to an iMessage number and get a finished, trend-informed Reel back in the same thread.

Sticky project preview