Team Delusion

The problem 60% of Indian farms sit outside reliable network coverage. Every agri-tech app assumes connectivity, a smartphone-literate user, and English — three assumptions that fail for the smallholder farmer standing in a field with a diseased crop. And when they do reach an expert, language is the next wall: the farmer speaks Kannada or Hindi; the agronomist often doesn't. What we built Mitti-Agent Live is one agent with two brains, sharing one case. Online — the live field walk. The farmer walks their field with the camera on, speaking naturally in their own language. A continuous Gemini Live session watches and listens: it answers in the farmer's language, and it interrupts unprompted when it spots crop damage the farmer never mentioned — proactive vision, not turn-based chat. When escalation is needed, a KVK agricultural expert joins the same session from a browser console: the expert speaks English, the farmer hears their own language, live, two-way — while the expert watches the farmer's camera feed and a structured case file (observations, urgency, requested items) builds itself from the agent's tool calls. The load-bearing trick — the case-state mirror. Every observation the cloud agent logs is persisted to the farmer's phone the moment it happens. The network dying is not an error path; it's a handoff. Offline — Gemma takes over. When the signal drops, the phone detects it in seconds, announces the switch, and hands the mirrored case to Gemma 4 E2B running fully on-device (LiteRT-LM). Gemma — with on-device vision — inspects the crop photo, understands the farmer's spoken observation in their language, triages the risk, generates safe next-steps and a required-items list, validates its own output (schema, risk code, 160-char reference packet), and drafts a complete field report as an SMS to the KVK. The farmer confirms before anything is sent. If the model itself fails, a deterministic safety layer keeps the loop alive — and the UI always says which brain answered. The safety boundary: Mitti never names a pesticide or a dose, never diagnoses with false certainty, never sends a message on its own. It routes evidence; the human expert decides treatment. On ambiguous input it defers instead of guessing. How we built it Android app (Java/Kotlin): continuous echo-cancelled mic streaming, Camera2 frames at 1 fps, live captions, network watchdog, case mirror, offline handoff — plus on-device Gemma 4 E2B via LiteRT-LM with the vision executor enabled, running text + image triage in a single inference pass. Gateway (Python/FastAPI): hosts one Gemini Live session per case; farmer and expert both connect to it, never to each other. Proactive audio (v1alpha), tool calling (log_observation, flag_urgency, request_expert), speaker-tagged audio for two humans in one session, camera relay to the expert console, and automatic Live-session reconnection with case-context replay so a cloud hiccup never kills the field session. Expert console (web): push-to-talk English, live translated conversation, the farmer's camera view, and the case file updating in real time. Why it matters for India Rural connectivity, language diversity, and low text literacy are treated as product constraints, not edge cases. Voice-first in 7+ Indian languages, vernacular↔English expert bridging, evidence that survives dead zones, and escalation over SMS — the one channel that works everywhere. The same architecture extends to ASHA health workers, veterinary triage, and rural repair — any field profession where the network is unreliable but the stakes are not.