Team SceneTrace
Gemini 4 NYC HackathonPoint your camera at something, or paste a YouTube link, and SceneTrace finds the exact product and lets you buy it on Shopify by voice. If the thing you're holding is broken, it tells you whether it's worth fixing and what the part costs. How it works - Seeing: Gemini 4 Hydrogen reads the photo or video and names each product: brand, model, colour, visible damage. - Researching: a Gemini managed agent looks up the exact model and retail price. It runs alongside the Shopify search, then re-ranks the offers when it finishes. - Ranking: Shop MCP returns live listings, and the Gemini Decisions API scores each one against what's in the picture, so the top offer is the right product. - Repair or replace: Hydrogen writes about 18 yes/no fault checks for the item. The Decisions API scores them against the photo, and Shop MCP prices a part against a new unit. A second Decisions pass rejects parts that don't fit, and Carbon writes the verdict. What you say counts too: "it won't charge" matters even when the photo looks fine. - Voice: it's fully hands-free. "What is this?", "the one from Craftedyogi", "the cheaper one", "number five", "search for the sugar-free one", "is it worth fixing?", "add it to my cart". The Decisions API routes each sentence. Before buying, it outlines the offer and asks "Add this one?". You can talk over it, and it ignores its own voice. - Checkout: one bag, with a checkout link per store. Where it breaks - Speech-to-text garbles store names, so matching is fuzzy, with numbered offers as the fallback. - Photo reading took 20–43 s. Low thinking brings it to 7–9 s (15–25 s to first offers), but the model is sometimes named generically until the agent corrects it. - Repair verdicts still vary run to run. When no part fits, it says so instead of guessing. Built with: Gemini 4 (Hydrogen, Neon, Carbon), Gemini Decisions API, Gemini managed agents, Gemini TTS, Shop MCP, Next.js.





