# XR Wave Agent pipeline to create visionPro + weber creatives

- **Event:** [Zero to Agent: Vercel x Deepmind Hackathon SF](https://cerebralvalley.ai/e/zero-to-agent-sf)
- **When:** Sat, Mar 21 at 9:00 AM – 10:00 PM (PDT)
- **Where:** Shack15, San Francisco, CA
- **Team:** [Romy Ilano](https://cerebralvalley.ai/u/romyilano)
- **GitHub:** https://github.com/romyilano/vercelDeepmind_2026-03-21
- **Gallery:** https://cerebralvalley.ai/e/zero-to-agent-sf/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/zero-to-agent-sf/hackathon/gallery/60

--                                                                                                     
  Hackathon Summary: Cross-Platform XR Scene Generator                                                    
                                                                                                          
  What We Built                                                                                         
                                                                                                          
  A natural-language-to-3D pipeline that takes English prompts and renders immersive scenes on both       
  visionOS (Apple Vision Pro) and WebXR (any browser) simultaneously, powered by a Gemini autonomous      
  agent.                                                                                                  
                                                                                                        
  "Place a surfer witch and a punk girl in an ocean scene"
    → Gemini agent (observe/reason/act/evaluate loop)
    → scene.json                                                                                          
    → RealityKit (.usdz) + A-Frame (.glb)
    → Two platforms, one command                                                                          
                                                                                                        
  Key Innovation: Agent, Not API Call                                                                     
                                                                                                        
  The Gemini integration isn't a single-shot API call — it's an autonomous agent loop:                    
                                                                                                        
  1. Observe — scans the filesystem for available 3D models and backgrounds                               
  2. Reason — calls Gemini with the prompt + asset context to produce a grounded plan                   
  3. Act — validates assets exist, swaps in fallbacks for missing ones, writes scene.json                 
  4. Evaluate — checks output validity, retries with adjusted prompts (up to 5 times)                     
                                                                                                          
  This means the agent adapts to what's actually available on disk rather than hallucinating assets that  
  don't exist.                                                                                            
                                                                                                          
  What It Does                                                                                          

  - Single command (./run.sh "your prompt") handles environment setup, agent execution, and dual-platform 
  output
  - Multi-character scenes — place multiple 3D characters with precise positioning                        
  - 360° backgrounds — equirectangular panoramas wrapped as immersive sky spheres                         
  - Batch mode — generate_all.sh auto-discovers every model in assets/models/ and builds a scene with all
  of them, no prompt or API key needed                                                                    
  - Logical asset mapping — one name ("bruja") resolves to .usdz for RealityKit or .glb for WebXR

---

Markdown version of https://cerebralvalley.ai/e/zero-to-agent-sf/hackathon/gallery/60. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt