Skip to Main Content

Agent University

Built at AI Engineer World's Fair Hackathon 2026 · Jun 27, 2026 · San Francisco, CA

Agent University — Demo video

Agents can already modify prompts, write tools, store memories, and run experiments. But without rigorous infrastructure, self-improvement is hard to trust. An agent may appear to improve while actually overfitting to a visible benchmark, reward-hacking the evaluator, or introducing hidden regressions. We solve this with three components. First, we provide an MCP trajectory recorder. It plugs into agent systems and captures the full experimental trace: prompts, tool calls, code edits, terminal commands, observations, metrics, failures, and final outcomes. Second, we provide skill versioning. Every prompt, tool, workflow, or agent skill can have versions. You can inspect what changed, compare performance, and see the trajectory of improvement over time. Third, we provide the harness for task-derived private held-out evaluation. The user creates the task, and Agent University helps build private held-out variants, seed splits, evaluator checks, and regression cases from that task. New skill versions are tested on those hidden cases without exposing them to the agent. This lets us detect reward hacking and overfitting before a skill is promoted. Together, these components create Agent University: a collective learning system for agents. When one agent solves a task, we do not simply copy its final prompt. We preserve the entire trajectory, distill it into a reusable skill, validate it on hidden tests, and transfer it to future agents. In our Flappy Bird control demo, a baseline agent needed around 100 experimental iterations to find a stable strategy. With Agent University, that experience is converted into a verified skill, and a fresh agent solves a held-out version of the task in about 3 steps. The core idea is simple: agents should not start from zero every time. But agent learning must be verifiable. Agent University makes agent self-improvement measurable, reproducible, and safe.

Team