# Berstein Lens

- **Event:** [Google I/O Hackathon](https://cerebralvalley.ai/e/google-io-hackathon)
- **When:** Sat, May 23 at 9:00 AM – 10:00 PM (PDT)
- **Where:** Shack15, San Francisco, CA
- **Team:** [Kyle Orciuch](https://cerebralvalley.ai/u/korciuch)
- **GitHub:** https://github.com/edgetrace/bernstein-lens.git
- **Demo video:** https://github.com/edgetrace/bernstein-lens/demo.mp4
- **Gallery:** https://cerebralvalley.ai/e/google-io-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/google-io-hackathon/hackathon/gallery/117

Bernstein-Lens is a gesture + voice control layer for AI-native developers. Named after Leonard Bernstein — who conducted a full orchestra without ever touching an instrument — it lets you control Cursor IDE and your entire OS using only your hands and voice.
Phase 1 is a MediaPipe hand-tracking pipeline running at 60fps with <12ms latency. Gestures map directly to OS actions: point to move the cursor, pinch to click, peace sign to scroll, palm for right-click, and a 🤙 invoke gesture to summon the semantic layer.
Phase 2 is the semantic control layer. Invoke triggers a liquid-glass circular HUD at the cursor position. Dwell on the mic zone to issue a voice command — transcribed locally via whisper.cpp + Apple Neural Engine CoreML in ~150ms. The screenshot and voice command are analyzed by a local vision model (Moondream via Ollama) to determine intent, returning a structured action: keystroke, click, type, or managed_agent. Complex developer tasks ("run the tests and fix failures") route to Gemini Managed Agents in a persistent cloud sandbox.

---

Markdown version of https://cerebralvalley.ai/e/google-io-hackathon/hackathon/gallery/117. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt