Skip to Main Content

Nils_Solo

Built at Meta Global AI Developer Hackathon · Oct 26, 2026 · Remote

Demo video · youtube.com/…

Sightline is an iPhone app that guides a blind or low-vision person's hand to the thing they want. Finding your own water bottle, pill bottle or keys on a cluttered table is a daily problem. A screen reader can say what is in a photo, but it can't move your hand to it. With the iPhone worn on the chest, you say "find my water bottle." Sightline finds it in the camera view. If more than one could be it, it asks which one, describing each: "The white one on the left, or the black one on the right?" Then it talks your hand in with short directions and tones: "Right. Forward. Stop, something's in the way." When you pick something up, it checks that it is the right one. Under the hood it is a multi-step pipeline. Meta models hear, understand, ask and segment. ARKit and LiDAR place the object and the obstacles around it in 3D. A guidance loop on the phone tracks one fingertip 30 times a second and never waits on the network. A saved room scan remembers where things were and how they look, and is used only when the object is not in view. Everything was built from scratch during the hackathon: a native iOS app, a Swift core library with 259 unit tests, a simulator of synthetic users, and a log of real device runs. Every failure found on the device became an automated check. Scope: indoors, within arm's reach. It does not replace a cane or guide dog.

Team