Skip to Main Content

Pit Crew is an autonomous agent that makes a repository's CI faster, then proves the faster CI still catches every bug it caught before. The problem: slow CI wastes hours every day, and AI coding agents make it worse by opening more PRs. Asking an AI to "make CI faster" is dangerous. In our test, a single Gemini call deleted real tests or platforms in 6 of 6 rewrites. Faster CI that catches fewer bugs is worse than slow CI. How it works: the agent proposes; deterministic code judges. - Analysis profiles months of GitHub Actions history and finds the slowest step on the critical path. - Optimizer makes ONE change from a playbook (cache, parallelize, shard, dedupe, affected-tests). A static "never check less" guard rejects any change that drops a test, OS, version, trigger or failure recording, or hides failures (|| true, retries, continue-on-error). - Proof picks real past bugs and planted mutations the new CI must still catch. It records the failures on video and in screenshots, and Gemini compares baseline and candidate visually. - Dispatcher + Rater (plain code) run the old and new CI interleaved 5+ times on a fork, replay every bug, and keep a change only if it is statistically faster (p ≤ 0.05) AND catches 100% of bugs. Kept changes open a PR with a proof card. Evaluation vs. baselines: 306 labeled edits on 18 popular repos the guard never saw (Django, Pandas, Ruff, Astro…). - Single model call: 0% of weakening edits caught. - Simple diff rules: at most 33% caught, and the strict one wrongly blocked 30% of safe edits. - Pit Crew: 100% caught (226/226), 0% of safe edits blocked (0/80). Where it failed: we document 5 guard holes we found, plus one unsafe sharding recipe of our own, and what we changed. Runner queue time can't be fixed by editing a workflow, so Analysis reports it as uncovered instead of inventing a fix. The Pit Wall dashboard shows each cycle live.

Team

tarunyakkalaavery_jonesKalkidan+1

More from Gemini 4 NYC Hackathon

We design an agentic system to automatically create + manage intelligent AI opponents/NPCs in video games. The suite of agents analyzes user gameplay…

Adaptive NPCs project preview

Manually routing PCBs is time-consuming and cost-inefficient. Current best algorithms (freerouting, PCBRouter) score very low on PCBWorld's hardest PCB…

routr project preview

Heliocam is a camera app where Gemini controls the camera. It watches what you're pointing at, sets the ISO, shutter, focus, white balance and zoom for you…

heliocam project preview

Repairs and assembly fail because manuals are lost, vague, missing, or nonexistent, so people break things or throw them out. KLIK turns a photo of any object…

Klik project preview

Iron Pulse turns your laptop into a 3D boxing ring. Fight an adaptive AI rival in real time using just your webcam. Throw real punches and lean to dodge…

Iron Pulse Arena project preview

Text your raw footage to an iMessage number and get a finished, trend-informed Reel back in the same thread.

Sticky project preview