AgentGYM

Hidden-test gating catches overfit harness mutations in a frozen coding LLM (Meta muse-spark-1.1; 16 tasks; 369 independently graded runs). 3/3 grader-caught failures were false completions (visible tests pass, hidden tests fail). SubQ Snap 1.1 read 12 traces (42k tokens) in one call and identified error/cancel-path handling as the root cause (p=0.96). Meta muse-spark-1.3 proposed harness mutations. The best raised dev pass rate 15/18 → 17/18 (+11pp), but failed the promotion gate on unseen validation tasks. On a held-out fault-injection task (6 reps), it scored 5/6 vs baseline 6/6 at 1.25x tokens: the gate was right. Controls: random search 5/6 (1.72x), one-shot LLM redesign 6/6 (2.03x). Small held-out sample (1 task × 6 reps).