Aawaaz (आवाज़) Real-Time Voice Agent for Rural Welfare Access 430 million Indians are functionally illiterate. They qualify for government welfare schemes PM-KISAN, PMAY, Ayushman Bharat, Ujjwala but can't navigate the portals to claim them. Aawaaz fixes that through real-time multimodal voice interaction not a chatbot with a microphone stapled on. Aawaaz uses the Gemini Live API for continuous, interruption-aware Hindi conversation. The agent listens in real time, extracts profile fields mid-speech without waiting for turn completion, and responds with natural Hindi adjusting tone when the user sounds confused or hesitant. It reads vocal cues, not just words. Aawaaz has a hard privacy boundary baked into the agent loop. The moment a user speaks or scans sensitive PII Aadhaar number, bank account, IFSC code the agent automatically switches from Gemini cloud to Gemma 4 E2B running fully on-device. That data is processed locally, stored encrypted in SQLite, and never transmitted to any server. When the sensitive exchange is complete, the agent seamlessly returns to Gemini Live for the rest of the conversation. The user never has to think about this the agent handles the boundary autonomously. Local Tesseract OCR scans Aadhaar and ration cards entirely on-device. Once the profile is complete, Aawaaz opens the actual government portal pmkisan.gov.in, pmayg.nic.in and fills every visible form field autonomously via Playwright, stopping only at OTP the one action only the user can take. Tech stack: Gemini Live API · Gemini Flash · Gemma 4 E2B (PII + offline) · FastAPI · Playwright · Tesseract OCR · SQLite (encrypted) · Web Speech API
Team
Placement
Participant
