# Pipeline Medic

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Hitakshi Shirude](https://cerebralvalley.ai/u/Hitakshi02)
- **GitHub:** https://github.com/Hitakshi02/pipeline-medic
- **Demo video:** https://youtu.be/TK-vYyCD2ZQ
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4358

Pipeline Medic: insurance for your data pipeline

It’s 3 a.m. and a nightly data pipeline fails. The alert says a contract broke. The cause is two stages upstream: a partner renamed a column, order_ts to ordered_at. Someone has to read logs, run ad hoc SQL, write a patch and rerun the job. And if you hand that job to an AI, it can “pass” by dropping bad rows, hard-coding values or loosening a test. The pipeline goes green and the numbers are wrong.

What it does

Pipeline Medic is an on-call agent for broken data pipelines. When a run fails, it:

Investigates with read-only tools: logs, schemas, table profiles and SQL, every query logged with an id.
Finds the root cause. Every finding must cite a logged query.
Writes a patch in a sandbox. It can’t touch contracts or tests.
Replays the failed night with the patch and compares the output to golden results and hidden invariants it was never shown.
Hands the root cause, patch and evidence to a human engineer, who approves or rejects.

What makes it different

The checks live inside the tools, so the model can’t skip them. Four verifiers cover the plan, the root cause, the fix and the claims. The fix verifier replays the night and rejects cheat patterns, such as blanket row drops, swallowed errors and hard-coded values. The claims verifier is a Muse Spark judge that refuses explanations not backed by evidence. When a check fails, the agent sees what failed, never the answer. Ground truth sits outside the agent’s folder.

In my first real run on a schema-drift incident, the agent was refused twice for unsupported claims, attached evidence, passed all verifiers and went to sign-off. It took 31 turns and 13 queries.

How I built and tested it

Pipeline: a synthetic eyewear retailer in DuckDB, with data contracts as the detector.
Fault injector: nine labeled fault types, from schema drift and null spikes to join fan-out and timezone shifts. It also makes multi-fault nights and clean-night controls.
Agent: Muse Spark 1.3 through the Meta Model API, with 12 tools and a console UI to watch runs and approve fixes.
Bench: on five incidents with verification on, all five outcomes passed the hidden checks, including a clean night where the right answer was to change nothing. One fault type was mislabeled because of a harness bug, which I fixed.
Router: I fine-tuned Muse Glimmer 30B (LoRA on Fireworks) as a triage router that predicts an incident’s fault type. It scores 0.975 on a held-out set versus 0.35 for the base model, and 0.88 on validation incidents versus 0.82 for rules and 0.70 for Spark zero-shot. It ties the rules on dev at 0.91, and it answers in 0.8 s versus 18.6 s for zero-shot Spark. The confidence intervals overlap, and the router’s confidence isn’t informative yet, so cheaper routing is future work.

What’s next

Widen the bench beyond five incidents, calibrate the router’s confidence, and add real pipeline connectors (dbt, Airflow).

Code: github.com/Hitakshi02/pipeline-medic

## More from Meta Global AI Developer Hackathon

- [CapyVerse](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4355)
- [David Barry](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4356)
- [Glint Companion](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4357)
- [HaystackedAI](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4359)
- [AIHI](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4360)
- [Risheek Aiyar](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4361)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/4358. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt