# Spark Gate

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Jayanth Dolai](https://cerebralvalley.ai/u/Jayanth_Dolai)
- **GitHub:** https://github.com/demonjd2026-afk/spark-gate
- **Demo video:** https://youtu.be/If_7ALm4S4k
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1841

Spark Gate is a CI safety gate for Muse Spark model upgrades. It tells you whether it is safe to move your agent to a newer model version or a cheaper endpoint, using your own conversations, before the upgrade ships.

Teams run agents on a pinned model. When a newer version or a cheaper endpoint arrives, switching is a guess: the new model may skip a policy check, call tools in a different order, change a refund amount, or stop returning valid JSON. Benchmarks can’t tell you that about your agent.

Spark Gate records real conversations of the agent on today’s model, replays them on each candidate through the Meta Model API, and measures exactly what changed: tool-call fidelity, policy rules, output schema, judged answer quality, latency and cost, each against a threshold you set. Muse Spark 1.3 judges quality in both orderings so position bias can’t decide, groups the regressions, explains them, and drafts a prompt fix that is re-tested on the whole suite before it counts. The verdict is cleared, cleared with fix, or held. As a GitHub Action it posts the report on the pull request and fails the check when the answer is no.

Results on the demo agent (a five-tool customer-support agent, 24 recorded conversations replayed 3 times each, 72 replays per candidate): Spark 1.2 cleared at 0.8x the cost of 1.1. Spark 1.3 was held for 2.1x p95 latency. Both contributor endpoints were over 90% cheaper but held, one for broken policy rules and invalid JSON, the other for 2.5x latency. The GitHub Action blocked a real pull request that tried to upgrade to 1.3.

The gate tests itself: 150 planted mistakes (skipped calls, swapped order, altered refunds, broken JSON, wrong outcome, forbidden calls) were all caught with zero false alarms on untouched copies, and the judge audit shows it never preferred the wrong customer’s answer and its two orderings agreed 36 of 36 times. Because Muse Spark is not deterministic even at temperature 0, single runs flipped a verdict; the gate now reports its own noise floor, pools three replays and scores quality as net loss.

Live dashboard: https://demonjd2026-afk.github.io/spark-gate/

Blocked pull request: https://github.com/demonjd2026-afk/spark-gate/pull/1

## More from Meta Global AI Developer Hackathon

- [Nikhil](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1838)
- [mucks](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1839)
- [Gol](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1840)
- [WorldKit](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1842)
- [Unmanually](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1843)
- [Trust Boundary](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1844)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/1841. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt