Skip to Main Content

Team SceneTrace

Gemini 4 NYC Hackathon

Point your camera at something, or paste a YouTube link, and SceneTrace finds the exact product and lets you buy it on Shopify by voice. If the thing you're holding is broken, it tells you whether it's worth fixing and what the part costs. How it works - Seeing: Gemini 4 Hydrogen reads the photo or video and names each product: brand, model, colour, visible damage. - Researching: a Gemini managed agent looks up the exact model and retail price. It runs alongside the Shopify search, then re-ranks the offers when it finishes. - Ranking: Shop MCP returns live listings, and the Gemini Decisions API scores each one against what's in the picture, so the top offer is the right product. - Repair or replace: Hydrogen writes about 18 yes/no fault checks for the item. The Decisions API scores them against the photo, and Shop MCP prices a part against a new unit. A second Decisions pass rejects parts that don't fit, and Carbon writes the verdict. What you say counts too: "it won't charge" matters even when the photo looks fine. - Voice: it's fully hands-free. "What is this?", "the one from Craftedyogi", "the cheaper one", "number five", "search for the sugar-free one", "is it worth fixing?", "add it to my cart". The Decisions API routes each sentence. Before buying, it outlines the offer and asks "Add this one?". You can talk over it, and it ignores its own voice. - Checkout: one bag, with a checkout link per store. Where it breaks - Speech-to-text garbles store names, so matching is fuzzy, with numbered offers as the fallback. - Photo reading took 20–43 s. Low thinking brings it to 7–9 s (15–25 s to first offers), but the model is sometimes named generically until the agent corrects it. - Repair verdicts still vary run to run. When no part fits, it says so instead of guessing. Built with: Gemini 4 (Hydrogen, Neon, Carbon), Gemini Decisions API, Gemini managed agents, Gemini TTS, Shop MCP, Next.js.

Team

utsavp257

More from Gemini 4 NYC Hackathon

We design an agentic system to automatically create + manage intelligent AI opponents/NPCs in video games. The suite of agents analyzes user gameplay…

Adaptive NPCs project preview

Manually routing PCBs is time-consuming and cost-inefficient. Current best algorithms (freerouting, PCBRouter) score very low on PCBWorld's hardest PCB…

routr project preview

Heliocam is a camera app where Gemini controls the camera. It watches what you're pointing at, sets the ISO, shutter, focus, white balance and zoom for you…

heliocam project preview

Repairs and assembly fail because manuals are lost, vague, missing, or nonexistent, so people break things or throw them out. KLIK turns a photo of any object…

Klik project preview

Iron Pulse turns your laptop into a 3D boxing ring. Fight an adaptive AI rival in real time using just your webcam. Throw real punches and lean to dodge…

Iron Pulse Arena project preview

Text your raw footage to an iMessage number and get a finished, trend-informed Reel back in the same thread.

Sticky project preview