AgentShield
Built at The Harness Engineering & Model Wrangling #SFTechWeek Hackathon · Oct 11, 2026 · San Francisco, CA

AgentShield is a self-evolving guardrail layer for agent-to-agent (A2A) traffic. When AI agents pass customer data to each other, off-the-shelf PII filters catch names and SSNs but miss company-specific identifiers like policy numbers and adjuster badges (0% recall at baseline). AgentShield sits between agents as a gateway. The inline path is fast and cheap: compiled regex rules plus Presidio NER, no LLM, ~16 ms. Asynchronously, an LLM auditor reviews forwarded traffic and records what leaked, storing only the shape, never the value. A synthesizer writes a new regex rule, and a validator gates it on hard metrics: recall on 300 labeled messages and false positives on 500 lookalike clean messages. A human approves in the terminal, and a MongoDB change stream hot-loads the rule into the live gateway with no restart. Every change writes a new eval generation (recall 0.51 → 0.75 in the demo, zero new false alarms). It also prunes itself: rules that drift into false positives across independent sessions are proposed for retirement, again with human approval. Recursive self-improvement that is measured, not claimed.