Skip to Main Content

World IR is an application that converts text prompts, images, and videos into interactive, editable 3D environments using Gemini AI models. The project addresses the limitations of traditional media, which is generally fixed to a particular viewpoint and difficult to modify or explore. Creating a 3D environment from existing images or videos typically requires manual modeling, animation, and scene reconstruction. World IR uses Gemini 4 to interpret input content and generate a structured description of the environment, including objects, spatial relationships, materials, lighting, and movement. The application converts this information into a persistent 3D scene that users can explore using free camera controls. For video inputs, the system uses computer vision to track moving objects and reconstruct their approximate trajectories in 3D. For image inputs, it uses visual references to guide scene layout and appearance. Text prompts can generate environments without requiring reference media. Users can navigate the resulting environment, change camera viewpoints, follow moving objects, and modify individual scene elements using natural-language instructions. Changes are applied to the existing scene without rebuilding unrelated objects. The project demonstrates how multimodal AI can convert static and recorded media into reusable, interactive 3D content for creative prototyping, visualization, education, and virtual production.

Team

anushka_naik

More from Gemini 4 NYC Hackathon

We design an agentic system to automatically create + manage intelligent AI opponents/NPCs in video games. The suite of agents analyzes user gameplay…

Adaptive NPCs project preview

Manually routing PCBs is time-consuming and cost-inefficient. Current best algorithms (freerouting, PCBRouter) score very low on PCBWorld's hardest PCB…

routr project preview

Heliocam is a camera app where Gemini controls the camera. It watches what you're pointing at, sets the ISO, shutter, focus, white balance and zoom for you…

heliocam project preview

Repairs and assembly fail because manuals are lost, vague, missing, or nonexistent, so people break things or throw them out. KLIK turns a photo of any object…

Klik project preview

Iron Pulse turns your laptop into a 3D boxing ring. Fight an adaptive AI rival in real time using just your webcam. Throw real punches and lean to dodge…

Iron Pulse Arena project preview

Text your raw footage to an iMessage number and get a finished, trend-informed Reel back in the same thread.

Sticky project preview