Team Recipe Forge

recipe-forge: recursive Nemotron agents that make NVIDIA's Nemotron serving recipes faster, losslessly, with every gain verified. It's basically Nemotron making Nemotron faster, while it learns each time! Every new Nemotron model ships with an NVIDIA vLLM serving recipe, and each one must be re-tuned to run fast. That takes scarce performance engineers, starting from zero each time. recipe-forge sends fleets of Nemotron agent teams (a Lead, Workers and a Verifier) at those recipes, one recipe and one GPU per mission: - They profile where GPU time actually goes. - They build a ledger from the model's architecture and attack the largest gaps. - They read and patch vLLM code, kernels and drafters, and freeze each candidate change into a sealed bundle. Nothing counts until a statistical gate verifies it: - interleaved AB/BA timing at alpha = 0.001 against measured null noise - a fresh reference re-measured on the spot - a lossless check: same tokens and logprobs within the band measured between two identical servers Version N+1, the recursion: - Each mission distills what it changed, on which architecture traits, and the outcome into a win library, failures included. New missions on related architectures start from it, and the harness tracks how often reuse helps. - A Mechanic rewrites the team's own playbook from the mission traces and keeps the new version only if it beats the old one on held-out recipes run side by side. Result: - On NVIDIA Nemotron 3.5 Bolt 74B with its official recipe (RTX PRO 6000 Blackwell, single stream), the agents found two changes on their own, each confirmed against a fresh reference: - --async-scheduling: +3.7% (325.5 vs 314.9 tok/s, noise 0.6%) - --attention-backend TRITON_ATTN: +3.4% We basically, in just one hour of running the agent, found two real, verified gains on the model NVIDIA released yesterday to make it faster! Who it's for: anyone serving open models (inference clouds, enterprises running Nemotron, NVIDIA's recipe teams). Repo: https://github.com/Hcoder10/recipe-forge