Trojans_VerdictLoop
Built at The Harness Engineering & Model Wrangling #SFTechWeek Hackathon · Oct 11, 2026 · San Francisco, CA
VerdictLoop - A Self-Improving Safety Harness for AI Agents AI agents are becoming increasingly autonomous, but giving them permission to modify infrastructure can introduce serious security risks. How can we safely allow agents to evolve their permissions without blindly trusting their decisions? VerdictLoop is a self-improving safety harness that evaluates proposed permission changes before they become active. Using a simulated infrastructure environment, VerdictLoop tests whether an agent should be allowed to perform actions such as restarting application services. It combines an eight-decision quick security probe with an independent, comprehensive safety gate. Safe permission changes can be promoted, while unsafe changes are rejected. **What makes VerdictLoop different is that it learns to test better.** When quick probes miss vulnerabilities that the full safety gate detects, VerdictLoop evaluates alternative testing strategies and persists validated improvements. In our synthetic held-out benchmark, the learned strategy detected all 3 distinct violating candidates, compared with just 1 before learning, using the same eight-decision budget per trial. **MongoDB Atlas** serves as the persistent control plane, storing policies, approvals, evaluation evidence, versioned strategies, and active state across application restarts. **Raindrop Workshop** provides visibility into real evaluator failures. We captured and inspected a case where the quick probe missed an alias inconsistency, but the complete safety gate correctly rejected the unsafe change. Workshop traces are linked to the corresponding MongoDB trials, making failures reproducible and inspectable. VerdictLoop demonstrates how AI safety infrastructure can become more effective over time without allowing the agent to bypass independent security checks.