Second Grader

Second Grader: trust, but verify your coding agent Coding agents describe their own work, and that description is often wrong. An agent will say "all tests pass" when it never ran them, or "only changed parser.py" when it also edited the tests to make them pass. Today a reviewer has to take the agent's word for it, or reread everything the agent did. Second Grader checks what the agent says against what it did, before anything merges. 1. Recorder: Hooks on Meta's Muse Code log every prompt, tool call, tool output, diff and test run to a trace. Each record is SHA-256 hash-chained to the one before it and Ed25519-signed, so editing, deleting or reordering any record breaks the chain at that exact record. Tool outputs are labelled OBSERVED (evidence), and the model's own words are labelled CLAIMED (what gets checked). 2. Auditor: Pulls checkable claims out of the agent's messages and tests each one against the OBSERVED records with plain code, not an LLM. It also catches test gaming: weakened asserts, added skips, deleted tests, special-cased inputs and changes to test settings. 3. Blinded verifier: A different model family (Gemini) writes its own tests from the task spec and the diff. It never sees the agent's messages or the agent's tests. 4. Gate: Merges the change if everything holds up. If not, it blocks the merge, quarantines the work, and shows a single verdict page that cites the record number contradicting each false claim. LLMs only pull out claims and write tests. Code makes every verdict. In the demo, the honest task merges with every claim green, every "trap" task (where the agent cheats) is blocked with the proof cited, and editing one record live turns the chain red from that point on.