XR Wave Agent pipeline to create visionPro + weber creatives

-- Hackathon Summary: Cross-Platform XR Scene Generator What We Built A natural-language-to-3D pipeline that takes English prompts and renders immersive scenes on both visionOS (Apple Vision Pro) and WebXR (any browser) simultaneously, powered by a Gemini autonomous agent. "Place a surfer witch and a punk girl in an ocean scene" → Gemini agent (observe/reason/act/evaluate loop) → scene.json → RealityKit (.usdz) + A-Frame (.glb) → Two platforms, one command Key Innovation: Agent, Not API Call The Gemini integration isn't a single-shot API call — it's an autonomous agent loop: 1. Observe — scans the filesystem for available 3D models and backgrounds 2. Reason — calls Gemini with the prompt + asset context to produce a grounded plan 3. Act — validates assets exist, swaps in fallbacks for missing ones, writes scene.json 4. Evaluate — checks output validity, retries with adjusted prompts (up to 5 times) This means the agent adapts to what's actually available on disk rather than hallucinating assets that don't exist. What It Does - Single command (./run.sh "your prompt") handles environment setup, agent execution, and dual-platform output - Multi-character scenes — place multiple 3D characters with precise positioning - 360° backgrounds — equirectangular panoramas wrapped as immersive sky spheres - Batch mode — generate_all.sh auto-discovers every model in assets/models/ and builds a scene with all of them, no prompt or API key needed - Logical asset mapping — one name ("bruja") resolves to .usdz for RealityKit or .glb for WebXR

Team

Placement

Participant