SightSense
Gemini 4 NYC HackathonAn AI guide for visually impaired and low-vision people, that sees the way ahead and talks you through it. The problem Who doesn't want to see the world around them? The face of a friend across the room, what's happening among us, the sign that points the way home. For at least 2.2 billion people living with vision impairment, much of that world arrives late, or not at all. A white cane finds what's at your feet, but not the branch at head height, the open cabinet door, or the bicycle coming fast. A guide dog helps, but only a small fraction of blind people have one: around 2% by common estimates. So the simplest moments, like reading a sign, finding the exit, or knowing who just walked in, still mean waiting, guessing, or asking a stranger. Everyone deserves to move through the world with confidence and know what's around them, not just get by. That's why I built SightSense. What SightSense does - Warns before you reach it. The phone spots obstacles, drop-offs and things coming toward you, then beeps and buzzes before saying a word. Warnings work without internet. - Speaks only when it matters. Its noise filter uses gemini models and knows when you're walking, ignores clutter, and never repeats itself. - Talk to it like a person. "What's in front of me?", "Is there an empty seat?" Gemini Live watches through the camera and answers in about a second. - Takes you places. "Show me the way out" finds the door and guides you there. - Reads for you. It helps you aim at signs, menus and labels, then reads and translates them. - Knows your people. Save a friend once, and it chimes "Alex is in view" when they appear. - You control it by voice, volume buttons, AirPods, or a layout you can use by touch. Built with: on-device AI vision (object detection and depth) for safety, plus Gemini Live and Gemini 4 models for understanding, acting and speaking.





