TracePilot AI

TracePilot AI is the AI debugger that proves its fix, or tells you it can't. AI coding tools write patches in seconds, but a patch that makes the tests pass is not proof that the bug is fixed. TracePilot investigates a failing service from its evidence (logs, screenshots, a bug report), writes a test that reproduces the bug, patches the code in a locked-down Docker sandbox, and lets the test suite decide whether the fix works. A second Meta model then reviews the patch without seeing the investigation. When the real cause isn't in the code, such as an expired API key, it says "no code change needed" instead of patching. Results (real runs; raw data in the repo): - Muse Spark Open rejected all 8 test-gaming patches in our benchmark, and all 6 model-generated cheating patches in a follow-up study (12 of 12 reviews). Each one passes the visible tests. - Blind test on 7 real bugs fixed in public projects (click, packaging, tomlkit and others) between Aug and Oct 2026, with the developers' own tests hidden: 2 fixed. One patch passed our checks but failed the hidden tests; we disclose and explain it. - It fixed a checkout bug whose only evidence was a customer's screenshot. Try it live, no account needed: https://tracepilot-ai.vercel.app/examples (click Run live, about 2 to 4 minutes, or Watch run). Every result, including failures: https://tracepilot-ai.vercel.app/evidence Built during the hackathon (Oct 27 – Nov 4, 2026); BUILD_LOG.md in the repo records what was built and when. ShipSpace, one test target, is my own earlier project: only its planted bug and the evaluation are new.