foolers_57

AgentTrace: debugging and continuous explainability for multi-agent AI workflows. When a multi-agent system gives a wrong answer, the step that shows the error is rarely the step that caused it. A wrong value from a tool, or an outdated policy document, quietly flows through three more agents before it reaches the customer. AgentTrace finds the true origin, separates it from the errors that merely propagated, and proves the diagnosis by experiment. We built it around a realistic five-agent customer-support workflow (Router → Retriever → Tool Agent → Policy Agent → Writer) and a labelled dataset of 104 scenarios covering nine fault types: wrong or empty retrieval, wrong or malformed tool data, bad decisions, hallucinated claims, two-fault runs, silent faults, and clean controls. Every scenario was verified to actually break the outcome, run three times with different seeds. The pipeline: Record and replay. Every agent, model, tool and search call is traced as a causal graph. Any run can be replayed exactly: 401 of 401 stored runs replayed with zero differences and no model calls. Counterfactual patching. Replace one node's output and re-run only what depends on it. All 230 of 230 patched replays changed only the target and its descendants. Deterministic validators. Free checks with no LLM call caught every malformed tool reply (33/33) and every empty or irrelevant retrieval (30/30). Diagnosis. Judges score every node on two separate questions: are its inputs trustworthy? and is its output correct given those inputs? A causal graph walk then separates root causes from propagated errors, and hard cases escalate to a deep-reasoning model. The top-ranked root was correct in 15 of 16 faulty runs (13 at the exact node). Verification by experiment. Each candidate root is patched and replayed. It's confirmed only if the outcome actually recovers. Every root verified this way was a true root (10/10).