# Agent Reliability Cockpit

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Akash Bhargava](https://cerebralvalley.ai/u/ABWRLD999)
- **GitHub:** https://github.com/AkashB98/agent-reliability-cockpit
- **Demo video:** https://www.youtube.com/watch?v=j8V3h5K3jYA
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3019

I am a marketing data analyst; my day job is GA4 and Google Tag Manager implementations, and the recurring pain is setups failing silently — a purchase event that drops revenue with no error, an API key leaked into a commit, a tool call nobody can replay. I built the Agent Reliability Cockpit: an agentic auditor driven by Muse Spark (muse-spark-1.1) through the Meta Model API. The agent runs its own audit tools over a synthetic analytics setup — event schema checks, secret scans, replayability tests, PII scrubbing, dollar-impact math — and every fix it proposes is blocked by default behind an approval gate. A human approves; the harness, never the model, applies it. In the demo, exactly one of four fixes is approved and applied to a copy: recorded revenue for one synthetic purchase goes from $0.00 to $49.99 while the originals stay byte-identical.

I tested it the way I'd want any auditor tested. On a sealed fresh holdout it had never seen: 5/5 faults detected (strict grader 5/5, typed grader 5/5; precision 6/7, disclosed). Across 10 holdout runs: 5/5 detected every time. On a clean dataset, my pre-registered test FAILED — the harness couldn't accept an empty findings list, so the model filed a placeholder; I fixed the harness and a separate post-fix re-run passed clean, and the original FAIL stays on the record exactly as it happened. Total Meta Model API spend for the entire build: $1.1534 of the $150 credit, itemized in the repo. The Cockpit composes guardrail libraries I open-sourced before the event (approval gate, secret guard, trace replay, PII redactor, judge calibration, injection firewall, MCP tool server); everything else — the agentic loop, the graders, the sealed datasets, the approval moment — was built during the event and is listed in the repo's BUILT-DURING-EVENT.md.

One note for the judges: I built this as an agentic use-case — the agent autonomously drives its audit tools and proposes fixes as tool calls, and every action operates under a hard human-approval gate (the gate log is in the repo's runs/ folder).

## More from Meta Global AI Developer Hackathon

- [Kurosyss](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3016)
- [VibeForge](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3017)
- [Agent SI](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3018)
- [Nara Abroad](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3020)
- [Name It](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3021)
- [Ghosts of Arles](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3022)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3019. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt