RPG - Robotic Planning with Gemini

Deploying robots in real-world indoor spaces is fundamentally constrained by two problems: environments are unknown ahead of time, and robot morphologies dictate what actions are physically feasible. A humanoid can reach a high shelf; a quadruped cannot. A mobile base can sweep an open floor; it cannot navigate a cluttered corridor. Traditional planning stacks require pre-built maps, hand-coded action libraries, and per-robot engineering. RPG eliminates all three by using a single multimodal model (Google Gemini) to perceive, map, and plan — with embodiment-specific constraints enforced at every stage. The system operates entirely from photographs. No LiDAR, no SLAM, no prior map data. A user walks through an unseen room, uploads sequential photos, and RPG synthesizes a structured spatial understanding, a generated floor plan, and a validated task plan — all conditioned on the selected robot morphology.