# Pit Crew

- **Event:** [Gemini 4 NYC Hackathon](https://cerebralvalley.ai/e/gemini-4-nyc)
- **When:** Sat, Nov 21 at 9:00 AM – 10:00 PM (EST)
- **Where:** New York, NY
- **Team:** [Tarun Sai Yakkala](https://cerebralvalley.ai/u/tarunyakkala), [Avery Jones](https://cerebralvalley.ai/u/avery_jones), [Kalkidan Reda](https://cerebralvalley.ai/u/Kalkidan), [Md Morsalin Mahmood](https://cerebralvalley.ai/u/mdmorsalin)
- **GitHub:** https://github.com/aj604/pit-crew
- **Demo video:** https://youtu.be/3WaO_D_Fa88
- **Gallery:** https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/88

Pit Crew is an autonomous agent that makes a repository's CI faster, then proves the faster CI still catches every bug it caught before.

The problem: slow CI wastes hours every day, and AI coding agents make it worse by opening more PRs. Asking an AI to "make CI faster" is dangerous. In our test, a single Gemini call deleted real tests or platforms in 6 of 6 rewrites. Faster CI that catches fewer bugs is worse than slow CI.

How it works: the agent proposes; deterministic code judges.
- Analysis profiles months of GitHub Actions history and finds the slowest step on the critical path.
- Optimizer makes ONE change from a playbook (cache, parallelize, shard, dedupe, affected-tests). A static "never check less" guard rejects any change that drops a test, OS, version, trigger or failure recording, or hides failures (|| true, retries, continue-on-error).
- Proof picks real past bugs and planted mutations the new CI must still catch. It records the failures on video and in screenshots, and Gemini compares baseline and candidate visually.
- Dispatcher + Rater (plain code) run the old and new CI interleaved 5+ times on a fork, replay every bug, and keep a change only if it is statistically faster (p ≤ 0.05) AND catches 100% of bugs. Kept changes open a PR with a proof card.

Evaluation vs. baselines: 306 labeled edits on 18 popular repos the guard never saw (Django, Pandas, Ruff, Astro…).
- Single model call: 0% of weakening edits caught.
- Simple diff rules: at most 33% caught, and the strict one wrongly blocked 30% of safe edits.
- Pit Crew: 100% caught (226/226), 0% of safe edits blocked (0/80).

Where it failed: we document 5 guard holes we found, plus one unsafe sharding recipe of our own, and what we changed. Runner queue time can't be fixed by editing a workflow, so Analysis reports it as uncovered instead of inventing a fix.

The Pit Wall dashboard shows each cycle live.

## More from Gemini 4 NYC Hackathon

- [State of the Art](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/85)
- [Dry Run](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/86)
- [Seen IT](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/87)
- [SmartCart](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/89)
- [Ledgerer](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/90)
- [3D SHOP](https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/91)

---

Markdown version of https://cerebralvalley.ai/e/gemini-4-nyc/hackathon/gallery/88. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt
