# Show Me

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Nirup Trivedi](https://cerebralvalley.ai/u/Nirup)
- **GitHub:** https://github.com/nirup2721/show-me
- **Demo video:** https://www.youtube.com/watch?v=3T2ld5rqNzk
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3774

Show Me is a phone web app that teaches you how to use any physical object in front of you — hands-free, with the camera checking your work.

TRY IT: open https://askshowme.com on a phone. If the browser shows a prompt asking for an access key, just tap Cancel — no key is needed during judging.

Point your phone at a machine, a panel, a piece of equipment. Hold the button and ask out loud: "how do I inspect this?", "show me how to use this machine". Show Me identifies the object, plans the steps, outlines the exact part you need on the live camera view, and speaks step one. Then it watches. Every couple of seconds it checks the camera frame against the step, and only when it actually sees you've done it does it move on — no tapping, because your hands are busy. If it can't tell, it says so and asks you to move closer. At the end you get a record of which steps were confirmed by camera.

Why I built it: I manage two gas stations with convenience stores in North Carolina. Every new hire asks the same questions — how do I use the coffee machine, how do I check the fire extinguisher, which breaker resets the register — and I can't always be there. Written manuals don't get read. The same problem exists in every small business, workshop, farm and home in the world: the knowledge of how to operate the thing in front of you is locked in someone else's head.

What is built and working:
- Identify + plan: Muse Spark 1.3 (vision) names the object and writes a short step list, each step with a one-sentence "what the camera should see when this is done".
- Outline: SAM 3.1 segments the part for the current step; the outline is drawn over the live camera.
- Voice in / voice out: Muse Voice Transcribe for the question; the phone speaks each step.
- Hands-free verification: a separate Muse Spark 1.3 verifier call (the agent cannot influence it) judges each frame as done / not yet / can't tell, with an evidence sentence. Two consecutive passes advance the step. A 60-second timeout hands control back to the user.
- Look-around mode: ask a general question and it surveys the scene — lists every machine in view, outlines each one, asks which you need.
- Open weights: Muse Spark Open (via Baseten) runs in parallel on every identify call and its answer is logged next to Spark 1.3's.
- Run logs, a run-state machine (identifying → guiding → verifying → needs_input → done), and a cost logger with a hard spending stop.

Tested on real equipment in a real store: fire extinguisher inspection (5 of 5 steps confirmed by camera), coffee machine (4 of 4, including waiting for the cup to fill by reading the percentage on the machine's own screen), slush machine, fuel pump, breaker panel. The honest limits are in the README: steps that happen outside the camera's view (hanging a fuel nozzle back behind you) can't be verified and fall back to a manual confirm.

Built solo in nine days. Python server on Modal, phone page on Vercel, MIT licensed.

## More from Meta Global AI Developer Hackathon

- [BOLUS](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3771)
- [Lisovate](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3772)
- [Abra](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3773)
- [Prysm](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3775)
- [CodeVision](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3776)
- [Rust Ghost](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3777)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3774. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt