Skip to Main Content

Every new open model is useless until someone figures out how to run it well. Today, LLM infra engineers do that by hand, or prompt by prompt with a chat assistant that can’t run anything, can’t verify an answer, and forgets what failed. Inference engines expose hundreds of settings (vLLM documents 180+ flags, SGLang about 135), and one latency target can involve 10,000+ configuration permutations. Good configuration is worth up to 40% on dense models and 50% on MoE models, and GPUs are scarce. Atlas GPU Harness is an autonomous LLM deployment agent. It runs open models on real inference engines, has a coding agent propose configurations, verifies every answer, memory peak and speed, and commits each result to MongoDB Atlas before the next step. Every new proposal starts from what Atlas returns. A setting is promoted only if it beats the default on held-out questions. Proven on real hardware: Kimi-48B: a 30 GB model served on a 16 GB GPU. The agent read its own failure from Atlas and rewrote its code (+30.73%), but the faster engine default was kept. Strata: a 68 GB model at 32K tokens. A +6.2% “faster” setting failed the answer check on 0 of 18 trials, so nothing was promoted. Qwen-35B: a +3.6% “speedup” failed confirmation and was rejected. Memory matters: with Atlas memory on, the agent needed 3 attempts instead of 4 and never repeated a known failure. Scale: 55 experiments across 11 runs, all exported from Atlas, crash-safe and resumable. The agent proposes, the harness decides what is true, and Atlas remembers everything. Next: vLLM and SGLang, then parallel agents tuning whole GPU fleets.

Team

henryl

More from The Harness Engineering & Model Wrangling #SFTechWeek Hackathon

MongoMation is an autonomous PLC engineer. You describe a machine in plain English, and Claude agents write IEC 61131-3 PLC code, compile and deploy it to a…

MongoMation project preview

LabRat is a self-improving harness for a physical lab agent, running on a real bench: two SO-101 arms, a 3018 CNC carrying a servo pipette I designed during…

LabRat project preview

Recursive Harness Scientist: an AI scientist that improves its own scientific method, and must replicate every gain on data it has never seen. Why it…

Recursive Harness Scientist project preview

Super Bid is a self-improving AI estimator that teaches itself to price bids the way a specific company would. It practices on the company’s real past bids…

SuperBid project preview

A robot that gets better at handling objects it’s never seen, without retraining. After every round, an AI coach reviews what broke and rewrites the robot’s…

WORLDSTATE gives physical spaces a memory. Any phone becomes an observer: open a link, point the camera, and WORLDSTATE tracks what's inside fridges, shelves…

WorldSTATE project preview