# AgentGYM

- **Event:** [AIE Code Summit 2026 Hackathon](https://cerebralvalley.ai/e/aie-code-2026-hackathon)
- **When:** Sat, Nov 7 at 9:00 AM – 10:00 PM (PST)
- **Where:** San Francisco, CA
- **Team:** [Jacob Valdez](https://cerebralvalley.ai/u/jvboid)
- **GitHub:** https://github.com/JacobFV/agentgym
- **Demo video:** https://youtu.be/DFY8fURg03g
- **Gallery:** https://cerebralvalley.ai/e/aie-code-2026-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/aie-code-2026-hackathon/hackathon/gallery/23

Hidden-test gating catches overfit harness mutations in a frozen coding LLM (Meta muse-spark-1.1; 16 tasks; 369 independently graded runs). 3/3 grader-caught failures were false completions (visible tests pass, hidden tests fail). SubQ Snap 1.1 read 12 traces (42k tokens) in one call and identified error/cancel-path handling as the root cause (p=0.96). Meta muse-spark-1.3 proposed harness mutations. The best raised dev pass rate 15/18 → 17/18 (+11pp), but failed the promotion gate on unseen validation tasks. On a held-out fault-injection task (6 reps), it scored 5/6 vs baseline 6/6 at 1.25x tokens: the gate was right. Controls: random search 5/6 (1.72x), one-shot LLM redesign 6/6 (2.03x). Small held-out sample (1 task × 6 reps).

## More from AIE Code Summit 2026 Hackathon

- [Apex Studio](https://cerebralvalley.ai/e/aie-code-2026-hackathon/hackathon/gallery/20)
- [Puddles](https://cerebralvalley.ai/e/aie-code-2026-hackathon/hackathon/gallery/21)
- [ballast](https://cerebralvalley.ai/e/aie-code-2026-hackathon/hackathon/gallery/22)
- [rebrowse](https://cerebralvalley.ai/e/aie-code-2026-hackathon/hackathon/gallery/24)
- [Flo](https://cerebralvalley.ai/e/aie-code-2026-hackathon/hackathon/gallery/25)
- [Rollback Club](https://cerebralvalley.ai/e/aie-code-2026-hackathon/hackathon/gallery/26)

---

Markdown version of https://cerebralvalley.ai/e/aie-code-2026-hackathon/hackathon/gallery/23. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt