# Sherlocked

- **Event:** [Gemini 4 NYC Hackathon](https://cerebralvalley.ai/e/gemini-4-nyc)
- **When:** Sat, Nov 21 at 9:00 AM – 10:00 PM (EST)
- **Where:** New York, NY
- **Team:** [Subhradeep Acharjee](https://cerebralvalley.ai/u/subhradeep), [Tanmay Sahu](https://cerebralvalley.ai/u/Tanmay3)
- **GitHub:** https://github.com/7dracoder/Sherlocked
- **Demo video:** https://youtu.be/EvWLcRuhccA
- **Gallery:** https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/22

Sherlocked is an iPhone app that works with Ray-Ban Meta smart glasses. It watches the other person's face at 24 frames per second, listens to their words and tone of voice, and quietly whispers a helpful cue into your ear, but only when it's actually warranted.

The problems it solves:

First, most video AI misses quick facial changes, or it pretends to be a lie detector. Sampling video at 1 frame per second misses movements that last less than half a second. In our tests it scored 0.00 on those, versus 0.57 to 0.86 at 15 to 24 frames per second. Many "AI lie detectors" also sound far too sure of themselves, mistake normal talking for tension, or interrupt honest conversations.

Second, glasses can't work with menus or constant chatter. If you have to look down at your phone to choose a mode or read a paragraph, you lose eye contact, which defeats the point.

How it works:

No mode picker, and it only starts when ready. It streams video and audio from the glasses (or from your iPhone camera and earbuds as a backup). It says "Sherlocked is listening" only after it confirms both video and sound are really coming in.
It figures out the situation by itself. In Two Truths and a Lie, it locks in its guess before the answer is revealed. In a deal conversation, it stays completely silent when people are being honest, and whispers a short question like "Ask what could delay Friday" only when the speaker's face tenses at a specific commitment. In practice mode, it holds its feedback until you finish, then shows a replay tied to exact frames.
It protects itself from bad data. It ignores frames affected by low light, tinted glasses, fast head turns, dropped frames, audio that's out of sync with video, people talking over each other, or the wrong person being in view. It also avoids mistaking an open mouth from talking for a lip press.

## More from Gemini 4 NYC Hackathon

- [Casita Lista](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/19)
- [AREA-Q](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/20)
- [Werewolf League](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/21)
- [Rohan Alexander](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/23)
- [CraftPOV](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/24)
- [Ghost Busters](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/25)

---

Markdown version of https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/22. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt
