Falden
Built at Meta Global AI Developer Hackathon · Oct 26, 2026 · Remote

Falden Receipts: your agent says it's fixed; Receipts tries to break it. Coding agents open pull requests that say "fixed" and "all tests pass", but the tests they cite can be ones the agent wrote or picked, so passing them says little about the claim. A wrong fix that passes its own tests merges, and the bug report arrives weeks later. A pandas PR written with Claude Code said all 24,928 groupby tests pass, and it merged. Twenty days later, a fix: empty categorical groups returned NaN. Receipts catches it. It found 9 blockers on the PR, each failing 5 of 5 on the PR and passing 5 of 5 on the code before it, in fresh containers; one of them also passes 5 of 5 on pandas' own later fix, and the other 8 were not checked against it. Receipts' fix loop then made six attempts, and none reached "survived fresh attack": the fifth passed the shown and held-out repros, and a fresh attack caught a performance regression in the fix itself, so Receipts does not call it fixed. Receipts read only the PR, but pandas' fix is public, so the models may have seen it. Receipts reads what the agent claimed, attacks the claims its budget covers with runnable tests in a sandbox with no network, and makes a finding a blocker only if it survives a skeptic: code gates (an assertion failure or a crash in changed code, failing 5 of 5 runs on the patch), surrogate fixes written without seeing the patch, and a different-model refuter. No runnable repro, no blocker. A blocker carries its repro and a record of how it ran. An operator can file it as a Linear ticket and post the result as a GitHub check that fails only on a confirmed blocker, and a receipt lists who checked what, at what cost, and what went unchecked. On a pre-registered SWE-bench Verified test split, with two template claims per agent patch (the issue is fixed; nothing else changed), it caught 56 of 135 bad or plausible-wrong patches (41.5%, 95% CI 33.5% to 49.9%), with false alarms on 33 of 90 good ones (36.7%, 95% CI 27.4% to 47.0%). Pre-existing, not used here: Falden's evidence pack and collector. Built during the hackathon, Oct 26 to Nov 4, 2026: Receipts' code, prompts and eval, and both demos, apart from credited third-party material.