Name: Backfill Tagline: A self-improving agent harness that writes GPU schedulers and beats the published state of the art on a real 6,212-GPU production trace. Problem GPU fragmentation. Fractional-GPU jobs leave slivers that no later job can use. Multi-GPU jobs need several free GPUs on one node, and some jobs require a specific GPU model. At full demand, hundreds of paid GPUs sit stranded. This is online bin packing with no migration, so every placement decision is permanent. What we built (problem statements: Recursive Harnessing and Long Horizon Engineering) Inner loop: gpt-4.1 writes a score(ctx) policy, the sandbox checks it, Atlas Vector Search catches duplicates, the simulator scores it on 4 workloads, and a critic writes lessons to Atlas. Outer loop: a meta-agent edits the harness spec (rules, context, tools, guardrails). Each version is saved to Atlas with its parent. A child is kept only if it beats its parent on held-out workloads. In our run, version 7 scored better in training but worse on unseen workloads, so it was rejected automatically. Results (GPUs left idle at 100% demand, lower is better) Production mix: Agent 311, GPU packing 387, FGD 452, best-fit 524 Beats FGD on 3 of 4 training workloads Still behind on jobs pinned to a GPU model. That’s the open gap the harness is working on. MongoDB Atlas Collections: harness versions with lineage, runs, lessons, workloads and a ledger. 3 vector indexes: Policy-code duplicates Lesson recall, filtered by family Workload fingerprint routing Also used: $graphLookup for harness lineage, aggregation for the leaderboard, change streams for the live feed, and Voyage embeddings through the Atlas AI Model API. Raindrop Every generation is a traced interaction, with each tool call as a span. Signals: beat_best, beat_fgd, regressed duplicate_policy, sandbox_violation, policy_crashed, guardrail_tripped harness_accepted, harness_rejected Self-edits are tagged by harness version. Stack: Python, OpenAI gpt-4.1, MongoDB Atlas plus Vector Search, Voyage AI, Raindrop, Starlette, and Vercel (dashboard reading Atlas live) Links: https://gpuoptimizer-dashboard.vercel.app/ https://youtu.be/T4KK97ZQODk
More from The Harness Engineering & Model Wrangling #SFTechWeek Hackathon
MongoMation is an autonomous PLC engineer. You describe a machine in plain English, and Claude agents write IEC 61131-3 PLC code, compile and deploy it to a…

LabRat is a self-improving harness for a physical lab agent, running on a real bench: two SO-101 arms, a 3018 CNC carrying a servo pipette I designed during…

Recursive Harness Scientist: an AI scientist that improves its own scientific method, and must replicate every gain on data it has never seen. Why it…

Super Bid is a self-improving AI estimator that teaches itself to price bids the way a specific company would. It practices on the company’s real past bids…

A robot that gets better at handling objects it’s never seen, without retraining. After every round, an AI coach reviews what broke and rewrites the robot’s…
WORLDSTATE gives physical spaces a memory. Any phone becomes an observer: open a link, point the camera, and WORLDSTATE tracks what's inside fridges, shelves…
