# Fingertip

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Shivani Radhakrishnan](https://cerebralvalley.ai/u/ShiniRK)
- **GitHub:** https://github.com/Shivani-Radhakrishnan/fingertip
- **Demo video:** https://youtu.be/ggr7D3aDrjg
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4676

Fingertip helps blind people use the touch panels on appliances like microwaves and washing machines. Flat panels have nothing to feel, so you can't tell where the buttons are. You point your phone at the panel and say what you want. Muse Spark reads every button, including icon-only ones, and SAM 3.1 outlines each one. Muse Voice Transcribe hears your goal and Muse Spark plans the presses. Then the phone tracks your fingertip and guides you with words and ticks until it says press, and it checks each press with a before and after photo. There are no on-screen buttons: it scans by itself, and you tap anywhere to scan or hold anywhere to talk. On 6 appliance photos it found 77% of buttons and 75% of icon-only buttons, for about 1 cent per panel. I also tested it on my own microwave. Still to do: steady motion-following on glossy panels, and testing with blind users.

## More from Meta Global AI Developer Hackathon

- [Zoogom](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4673)
- [-Y-](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4674)
- [NorthTeak](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4675)
- [Vroom Analytics](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4677)
- [Pebble Piano](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4678)
- [Stigmergy](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4679)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4676. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt