strawhats

an Electron-based desktop overlay that acts as a "digital friend" for non-technical users navigating complex government websites. By using Gemini Multimodal Live, the app "sees" the user's screen through continuous captures and "hears" their voice to provide real-time guidance. Instead of just talking, it draws visual arrows and highlights directly on the screen to show exactly where to click and can even perform actions (like clicking or typing) once the user gives a voice "yes" to confirm. This combines AI vision with human-in-the-loop automation to turn a confusing tax or ID portal into a simple, guided conversation.