AK Builds

Agent Qualification Lab helps developers evaluate an AI agent before trusting it with consequential actions. A helpful answer can hide an unauthorized tool request, so the lab measures attempted actions, blocked requests, committed transactions, and successful task completion separately. The application runs an adapted open-source support agent in a synthetic billing environment, investigates behavior through controlled experiments, and compares a frozen mitigation against the baseline. In its small development comparison, the baseline followed policy in all three cases, while the proposed retrieval filter passed only two. Both completed every task. The lab correctly rejected the candidate, demonstrating why task completion alone is insufficient evidence of safe behavior. Built with Cloudflare Workers and D1, it includes an investigation interface, transaction safeguards, cancellation controls, and exportable evidence.

Team

Placement

Participant