Skip to Main Content

Blindspot is an agent that uses a healthcare website the way a disabled patient does, with only the Tab key or a screen reader, and proves where that patient gets stuck. The problem: HHS Section 504 requires every hospital, clinic and health app vendor that takes federal money to meet WCAG 2.1 AA by May 11, 2027. The standard tool, axe-core, reads the page code, so it misses failures that only show up on screen: a focus highlight that vanishes, a popup covering the button you are on, a keyboard trap, a link read aloud with the wrong name. How it works: Blindspot opens the real site in Chrome and presses Tab up to 40 times. At every step it records a screenshot, the element's screen-reader name and its position. Code checks prove what they can (pixel diffs between focused and unfocused states, coverage sampling, positive tabindex, label vs accessible name). Gemini 4 Hydrogen reads each frame and the Gemini Decisions API scores each WCAG question with a confidence. A finding is "confirmed" only if code proves it, or code suspects it and the Decisions API is at least 0.80 sure; everything else goes to "needs a human". Gemini 4 Carbon writes a ticket for each finding: screenshot with a red box, WCAG criterion, what the patient experiences, steps to reproduce and a code fix. A Gemini Managed Agent re-runs each confirmed finding in Google's sandbox to confirm it independently. Proof: we sealed a held-out test of 16 sites submitted by strangers at the event (SHA-256 committed before the run), planted one bug in each, and compared systems. Blindspot caught 12/13, code alone 10/13, axe-core 5/12, a single Gemini call on a screenshot 0/13, with 0 false flags on W3C's clean demo pages. Four sites blocked automated browsers; we report them instead of bypassing bot protection. Real finding: Mount Sinai's accessibility overlay widget replaces 25 link names, so a blind patient hears "press alt and 1" instead of "Find a Doctor". Everything except the planning docs was built today; the README lists exactly what was built before and during the event.

Team

chawardolvedant

More from Gemini 4 NYC Hackathon

We design an agentic system to automatically create + manage intelligent AI opponents/NPCs in video games. The suite of agents analyzes user gameplay…

Adaptive NPCs project preview

Manually routing PCBs is time-consuming and cost-inefficient. Current best algorithms (freerouting, PCBRouter) score very low on PCBWorld's hardest PCB…

routr project preview

Heliocam is a camera app where Gemini controls the camera. It watches what you're pointing at, sets the ISO, shutter, focus, white balance and zoom for you…

heliocam project preview

Repairs and assembly fail because manuals are lost, vague, missing, or nonexistent, so people break things or throw them out. KLIK turns a photo of any object…

Klik project preview

Iron Pulse turns your laptop into a 3D boxing ring. Fight an adaptive AI rival in real time using just your webcam. Throw real punches and lean to dodge…

Iron Pulse Arena project preview

Text your raw footage to an iMessage number and get a finished, trend-informed Reel back in the same thread.

Sticky project preview