Gemini Game Engine
Gemini 4 NYC HackathonGemini Engine rips the deterministic game engine out of 2D arcade games and replaces it with the Gemini Decisions running live in a real-time loop. There is zero hardcoded physics, zero collision math, and zero hardcoded AI in Python or JavaScrip. The browser Canvas is a pure dumb 32×18 pixel-grid painter, and neural networks run the entire game from creation to playtesting to live execution. How It Works (Prompt → Game → Testing → Playing): 1. Prompt: A user or judge describes any 2D game idea (e.g., Fruit Basket, Helicopter, Solo Squash) or loads a premade demo (Pong, Fruit Basket) — or live-tweaks an active game. 2. Game Creation (Gemini 4 Neon): Designs a structured 32×18 integer pixel-grid GameSpec (entities, controls, movement/collision rules, scoring) and compiles crisp monochrome pixel-art sprites. 3. Autonomous Multimodal Playtesting (Gemini 4 Hydrogen): Before handing the game to the human, an interactive Playtester agent actually plays the game tick-by-tick on gemini-decision-1 using function calls (press_key, reset_game, finish_playtest) while visually inspecting live rendered PNG frames streamed onto the Canvas. It deliberately tests adversarial edge cases (missing balls, hitting walls, colliding with hazards) and sends surgical bug reports back to Neon in a self-healing loop until approved. 4. Live Neural Game Engine (Gemini Decision 1): On every tick, gemini-decision-1 receives the live rendered Canvas frame (128×72 JPEG), entity state, rules, and player key, and evaluates a single batch of Score (7-bin integer delta dx/dy), Choice (state), and Noul (score, respawn, game_over) questions with 0 output tokens. Pushing Real-Time Loops & Where It Breaks: - Scaling Cliff: Batching speculative action trees jumped throughput from 3.0 FPS (6q, 331ms) → 5.7 FPS (24q, 350ms) → 8.4 FPS (60q, 359ms), before hitting a 2× prefill latency cliff at Depth-4 (160q, 726ms / 5.5 FPS). - Smooth Real-Time Rhythm: Pairing Depth-2 Speculative Trees with All-Branch Parallel Async Prefetch (asyncio.gather) and an Adaptive p80 Metronome Pacer (Δt = p80_RTT / 2) eliminates Wi-Fi burst-freeze stutter. - Proof It Works: Live Shadow-Mode Ground Truth overlay (real-time cell MAE) + reproducible headless eval (bench/eval_harness.py) improved Pong edge-case accuracy from 50% → 90% (+40 pp) and achieved 100% accuracy (0.00 MAE @ 10.9 FPS) on a 25-tick runner rollout. - Remaining Limits: Simultaneous compound inequalities in 0-CoT (0.36 confidence on Pong bottom-wall bounce), closed-loop bounce phase shifts, and high-state games like Tetris.





