# Expectrail

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Sean Keluo-Udeke](https://cerebralvalley.ai/u/seanudeke)
- **GitHub:** https://github.com/seanudeke/Expectrail
- **Demo video:** https://www.youtube.com/watch?v=BlWOiweL8Lc
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2378

Coding agents now change code faster than anyone can review it. The gate everyone relies on, "the tests pass", only checks what someone already wrote a test for, and asking the agent whether its fix worked lets it grade its own homework. Expectrail is an independent gate for changes an AI agent makes: it decides from what the changed software actually does, never from the agent's account.
How it works. A person approves each check as one plain-English sentence ("the player never falls through the ground"); Muse Spark can draft the test spec from it. After the agent's change, a harness replays the game on a fixed route, the same way every time, and captures every frame. SAM 3.1 finds what the check names in the frames, the game engine logs where things really are, and a deterministic evaluator, plain code rather than a model, compares the two. A missing or uncertain observation is "needs review", never a pass. Muse Code fixes bugs from a plain report in a sandbox where it can't reach the checks, the held-out cases or the signing key; if an attempt fails, the next one gets Expectrail's evidence. Only a change that passes every check, on public and private routes, gets a signed release candidate for a person to approve.
What it catches. On fifteen fixes with known answers, in three runs, Expectrail accepted all 8 correct fixes and none of the 7 that only look right. One fake fix stops the robot falling by drawing it inside the floor: the engine's position log accepted it, and Muse Code, shown before-and-after screenshots, said the fall was fixed. Claude Fable 5.1 and GPT-6 Astra, given the same screenshots, also accepted none of the fakes, but turned down correct fixes 5 and 1 times in 24. Expectrail caught 8 of 8 held-out bugs kept from the fixer with no false alarm, accepted 10 of 10 live end-to-end runs, and carried over to a web checkout page unchanged in its evaluator and decision rule (8 of 8 bugs, and 4 of 4 private bugs written by a separate agent).
What it can't do yet, measured. A pre-registered mutation bench shows the checks miss most changes nobody anticipated: of 77 random edits that differ from the clean game they caught 10 (0.9% weighted), mostly because no check covers scenery. Advisory discovery, which looks for problems nobody wrote a check for, flagged 7 of 16 fresh defects blind and no healthy build; for now it only advises. And Muse Code never wrote a fix that only looked right on bugs we didn't write, so there is no blind catch on a real-world bug yet.
Every number comes from a CI run's own output, recorded in the repository with the command that reruns it. Watch the video, try the interactive demo at expectrail.com/demo, or read the results and their limits in the README.

## More from Meta Global AI Developer Hackathon

- [BizMind AI Agency](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2375)
- [Fire Walkers](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2376)
- [TaskLens](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2377)
- [YSAI Nexus](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2379)
- [CircuitLens](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2380)
- [Time Space](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2381)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2378. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt