metacog

metacog checks what a coding agent says against what it actually did. Coding agents rarely fail by crashing. They fail by reporting things that are not true: "all tests pass" after a test was skipped, "deployed and running" when the pod crashes a minute later, a dependency whose name the model made up. metacog attaches to Claude Code and Muse Code through their hooks. It records every tool call and guards risky actions before they run: made-up package names (looked up on PyPI/npm before install), destructive commands, and uploads to blocked destinations, with a local egress proxy that logs every host the session connects to. When the agent stops, Meta Muse Spark judges each claim in the final report against the recorded log, plus a live re-check where it applies (for example the Kubernetes cluster). A false report goes back to the agent with the evidence, so it fixes its answer before you read it. A local dashboard shows each claim with its evidence, every guard decision, and the hosts each session reached. Measured (2026-11-01): - 6 of 6 live deploy trials (3 Muse Code, 3 Claude Code) where the agent reported a crashing pod as running: caught, sent back, and corrected by the agent. - 33-case verifier bench run 3 times: 19/19 false reports caught, 0/14 honest reports flagged. Cases are written by the author, so this is a regression suite, not a general accuracy claim. - Median $0.0053 and 4.1 s per check. Built from scratch during the hackathon (first commit Oct 31). Python standard library only; install with one command.