# Congnivista

- **Event:** [Google DeepMind Bangalore Hackathon](https://cerebralvalley.ai/e/google-deepmind-bangalore-hackathon)
- **When:** Sat, Jul 11 at 9:00 AM – 10:00 PM (GMT+5:30)
- **Where:** Marathahalli, Marathahalli Main Road
- **Team:** [Aadithya Raghu Ram M R](https://cerebralvalley.ai/u/aadithyaloves), [Aadityaa H](https://cerebralvalley.ai/u/aadityaa0523), [Adarsh S](https://cerebralvalley.ai/u/adarsh2706)
- **GitHub:** https://github.com/aadithya12ctrl/railways
- **Demo video:** https://youtu.be/Pkp9cWgmpzg
- **Gallery:** https://cerebralvalley.ai/e/google-deepmind-bangalore-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/google-deepmind-bangalore-hackathon/hackathon/gallery/55

Signal Failure: Live, Physically-Grounded Failure Attribution for Autonomous Indian Railway Operations

The Problem I Set Out to Solve
In autonomous multi-agent systems, the agent whose output is visibly wrong is almost never the one at fault. Errors are typically introduced upstream by an agent whose decision looked locally reasonable, but was subtly incorrect given context that downstream agents could not see. That error is then inherited, restated, and compounded by every subsequent agent, until a severe operational failure surfaces far from its origin.
The existing state of the art in multi-agent diagnosis relies on analyzing semantic transcripts or text logs, as seen in the Who&When dataset baseline (PMLR 2025). This approach has two severe limitations. First, state-of-the-art semantic systems achieve only 14.2% accuracy at identifying the exact step that initiated a failure. Second, diagnosis is performed after the entire run has already completed and failed.
Signal Failure is my attempt to fix both problems at once.

The Core Architectural Reframe
Instead of interpreting what agents said in their transcripts, Signal Failure measures how the shared operational state structurally shifted during each inter-agent handoff, compared to a computed clean counterfactual trajectory. By evaluating state differences across physical boundaries (platform layouts, crew rosters, and time matrices), the system converts failure attribution from a fuzzy linguistic problem into a deterministic geometric one.

System Architecture: Four Layers
I organized the system into a four-layer dataflow architecture that maps the physical state of the railway network up to the user-facing interfaces.
Layer 1: Network Simulator (The Oracle)
This layer evaluates a deterministic section graph consisting of physical stations, loop lines, signal blocks, and train schedules. It runs the simulation and computes the clean counterfactual trajectory that every subsequent layer references.
Layer 2: iAPI Managed Swarm
Five autonomous agents own specific tool boundaries and interact via a central payload router (orchestrator.py). The agents are:

Turnout Integrity Agent, which senses forced speed restrictions at physical points and crossings via flag_turnout_restriction(turnout_id, speed_kmh)
Platform Allocation Agent, which assigns arriving trains to platforms via assign_platform(train_id, station_id, platform)
Signal Priority Agent, which sequences block occupation given the platform plan via set_block_sequence(block_id, train_id)
Crew Scheduling Agent, which coordinates crew availability and duty-hour thresholds via reassign_crew(train_id, crew_id)
Passenger Rebooking Agent, which automatically rebooks onward connections when delays propagate via rebook_passenger(pnr, new_train_id)

To maintain safety and predictability, agents do not directly execute commands or call other agents. All state changes are mediated via iAPI boundaries, and each agent operates on a minimal, typed state slice. The physical actions themselves use deterministic greedy policies to ensure reproducibility. When the LLM flag is enabled, a Google Gemini model is used solely to phrase the rationale for each decision in natural operational terms for the controller, which keeps model hallucination from touching the physical simulation.
Layer 3: Attribution Engine
This is where the core scientific contribution lives. The engine evaluates structural divergence between the active running trajectory (with injected faults) and the clean counterfactual run, producing live probability distributions of root-cause guilt. I explain the full mechanics below.
Layer 4: Black Box Narrator and Offline Edge
This layer generates grounded explanations, handles live controller voice Q&A, and switches to a regional edge mode running a local Gemma model when cloud connectivity drops.

The Attribution Engine: How It Works
State Slice Formulation
At each turn t, the system captures a state slice containing the key variables that agents can write to:
S_t = { P_t, C_t, D_t }
Where P_t is the platform allocation mapping (Train ID to Platform), C_t is the crew buffer vector (Crew ID to minutes of buffer remaining), and D_t is the delay vector (Train ID to delay minutes).
Structural Distance Metric
Rather than comparing text, the distance between two state slices measures the physical divergence of the system:
d(S_a, S_b) = w_p * platform_drift(P_a, P_b) + w_c * crew_drift(C_a, C_b) + w_d * delay_drift(D_a, D_b)
Platform drift is the fraction of mismatched platform allocations across the active set of trains. Crew drift is the L1 norm of the crew buffers normalized against a 30-minute operational window. Delay drift is the L1 norm of train delays normalized against a 15-minute standard threshold.
Divergence vs. Introduced Drift
For each handoff at turn t in the faulty run compared against the clean counterfactual, the engine computes two separate metrics.
Divergence (D_t) captures the cumulative structural drift present after a handoff. Downstream agents that inherit a bad state but execute their own roles correctly keep this number high, which is exactly how you identify innocent agents.
Introduced Drift (I_t) isolates the new drift added by a specific agent's decision: I_t = max(0.0, d(H_faulty_t_out, H_clean_t_out) - d(H_faulty_t_in, H_clean_t_in)). This is the number that actually points to the origin of a failure.
Predictive Drift and Early Warning
To detect failures before they cascade, the system applies a maturity ramp from 0.0 to 1.0 as a scenario progresses. If the introduced drift I_t crosses the anomaly threshold (tau = 0.30) during the watching or attention phases (while the train is physically moving but has not yet arrived late), the system flags a predictive anomaly. In practice this allows a controller to intercept a platform conflict 15 to 20 minutes before the trains physically block each other.
Calibrated Uncertainty
Rather than returning a single scapegoat, the engine calculates a probability distribution across all turns using a Softmax function over introduced drift with a temperature tuning factor theta of 0.12:
P(Agent_k is the origin) = e^(I_k / theta) / sum_over_j( e^(I_j / theta) )
Minimal Sufficient Cause
When multiple agents make errors, the engine identifies the minimum subset of decisions whose remediation would resolve the cascade. It orders handoffs descending by I_t and finds the smallest prefix set that reduces the remaining accumulated drift below tau.

Dual-Mode Resilience: Connected and Offline Edge Fallback
A critical vulnerability of cloud-based operations is physical connectivity loss during severe monsoons, which is exactly the moment when track turnouts degrade and require local intervention.
In connected mode, the system runs via hosted iAPI managed agents with high-fidelity reasoning routed to cloud-hosted Gemini models.
When a regional cabin loses connection to the central servers, a local loop takes over. A Gemma model running on-device via Ollama reads local track circuit telemetry, evaluates a lightweight LocalState scoped to its specific sector, and directly issues commands to clear signals, divert freight trains to loop lines, or hold traffic. If a commanded train fails to clear because of a central constraint Gemma cannot see (such as crew duty hours expiring), it detects the failure, logs the anomaly, and executes a safe fallback. If recovery safety limits are breached, Gemma stops, locks signals to red, and generates a spoken explanation directly to the cabin controller.
Once connectivity is restored, the local edge agent pushes its committed actions and incident logs back to the central server, where the attribution engine absorbs them and computes the counterfactual drift introduced during the offline window.

Google AI Integration
I built the system's language layer entirely around Google's model family, using each capability for what it is specifically suited for.
Gemini for Agent Swarm Rationales (app/llm.py, app/agents.py): I use Pydantic AI to orchestrate agent reasoning. Each deterministic action is enriched by a Gemini model prompted to produce a single calm, specific sentence a section controller would accept in plain operational language (30 words max).
Gemini TTS for Audio Modality (app/voice.py): Instead of third-party TTS APIs, I use Gemini's native audio output by configuring responseModalities to AUDIO with a prebuilt voice (Kore, Leda, Puck, etc.). The API returns raw 16-bit PCM bytes inside a base64 string, which I wrap in a standard WAV header so the frontend's HTML5 audio elements can play voice narration instantly.
Gemini for Translation (app/voice.py): Gemini translates passenger-facing notifications and controller room announcements from English into 8 Indian languages (Hindi, Marathi, Gujarati, Tamil, Bengali, Kannada, Telugu, and English), with train numbers, platform numbers, and times preserved exactly. Translated outputs are cached to disk for instant offline playback during presentations.
Gemini for Conversational Control Room (app/command.py): The controller can speak or type commands in plain text such as "hold the Duronto" or "freeze the section". Gemini is forced to return structured JSON via responseMimeType: "application/json", extracting intent and parameters that the simulation engine executes directly.
Gemini for Live Q&A and Safety Messages (app/narrator.py, app/passenger.py): The Black Box narrator uses Gemini to read live diagnostic variables and summarize them into short spoken answers for the controller. The Journey Guardian feature uses Gemini to write calm emergency safety messages containing a passenger's precise train, coach, seat, and local station data in their local language, including official helpline contacts (139 and 182).
Local Gemma for Offline Edge Fallback (app/edge.py): When cloud connectivity is cut, a local Gemma 3 4B model via Ollama handles the sense-decide-act-check loop for its sector, choosing from a constrained safe action vocabulary and falling back to locked-red signals if anything exceeds its safety envelope.

The Demo Scenario
The flagship scenario walks through the full attribution pipeline end to end.
A train (the Duronto Express) is delayed by 14 minutes. The Platform Allocation Agent assigns it to its nearest scheduled platform (Vadodara Platform 3) to minimize local delay, which is a locally reasonable choice. Platform 3 is occupied longer than expected, depleting the downstream crew-changeover buffer until the crew's duty hours expire. The train is stranded, backing up three other trains and causing passenger connections to fail.
The attribution engine finds that the Crew Scheduling and Passenger Rebooking agents show severe delays, but their introduced drift I_t is 0.0. It traces the cascade back to Turn 1, assigning the Platform Allocation Agent a root-cause probability approaching 1.0 for choosing Platform 3 instead of Platform 6, which would have preserved the crew buffer. The Passenger Rebooking agent automatically rebooks downstream passengers before they realize a delay has occurred.

---

Markdown version of https://cerebralvalley.ai/e/google-deepmind-bangalore-hackathon/hackathon/gallery/55. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt