Skip to Main Content

Cinema Puppeteers

Gemini 4 NYC Hackathon

God-Shot turns one continuous film shot into a verified overhead blocking map, lets you change the camera, and re-shoots the scene from the new angle. Gemini 4 Argon watches the shot and writes a scene map: room, objects, actors with paths, and the camera’s position, direction and lens over time. Our code turns that map into dozens of geometric predictions (“at 0:03 the door should be in the left half”, “the camera pans left between 0:04 and 0:06”). The Gemini Decisions API checks every prediction against the real frames in batched yes/no and multiple-choice questions and returns calibrated probabilities. Argon corrects what failed. A correction is kept only if it scores better on the same shared question bank, so it can’t “improve” by deleting the hard questions. Up to 3 rounds; whatever still fails is flagged low-confidence on the map instead of being shown as fact. We render a 9:16 reel: the clip on top, the animated top-down map below, with “?” badges where it’s unsure. Then you direct: a camera board with POV presets (over-the-shoulder, reverse, close-up), drag/draw/orbit tools, 180° and 360° arcs, push-ins, a keyframe timeline and live FOV. Re-render the map instantly, or re-shoot with Gemini Omni: a map-exact grey-box guide plus reference frames produce the new shot, and director agents (scene analyst, director, critic) keep the actors’ performance and apply free-text requests. The same verifier then scores the generated video against your edited map, so you see how well Omni actually followed the camera direction. What raw video gives us that text can’t: the geometry. Direction, distance, timing and layout only exist in the pixels. Evaluation: headline accuracy is scored against human labels only, never by the verifier. Baseline (one Argon call) vs. the verify-and-fix loop on held-out clips we didn’t tune on: a CameraBench subset with expert camera-motion labels, open-licensed Blender films, and clips filmed by people outside our team. Prompts and thresholds were frozen before the test split ran. We report per-type accuracy, verifier-vs-human agreement, calibration, whether low-confidence flags are really wrong more often, and a failure gallery (whip pans, crowds, low light, long takes).

Team

sivabalanAkashgunaprajeshk+1

More from Gemini 4 NYC Hackathon

We design an agentic system to automatically create + manage intelligent AI opponents/NPCs in video games. The suite of agents analyzes user gameplay…

Adaptive NPCs project preview

Manually routing PCBs is time-consuming and cost-inefficient. Current best algorithms (freerouting, PCBRouter) score very low on PCBWorld's hardest PCB…

routr project preview

Heliocam is a camera app where Gemini controls the camera. It watches what you're pointing at, sets the ISO, shutter, focus, white balance and zoom for you…

heliocam project preview

Repairs and assembly fail because manuals are lost, vague, missing, or nonexistent, so people break things or throw them out. KLIK turns a photo of any object…

Klik project preview

Iron Pulse turns your laptop into a 3D boxing ring. Fight an adaptive AI rival in real time using just your webcam. Throw real punches and lean to dodge…

Iron Pulse Arena project preview

Text your raw footage to an iMessage number and get a finished, trend-informed Reel back in the same thread.

Sticky project preview