Bignesh Das

Can an AI model make its own inference faster, safely? Muse Glimmer 30B (Meta, 4-bit, MLX) runs on a Mac Studio M5 Max. I built a referee: a safety gate, a bit-exact correctness check against a golden reference, and replicated, interleaved A/B speed races with rules fixed before any result. With it, Glimmer now generates 5–21% faster on its own 19 recorded self-tuning prompts (median +12.8%), 17% faster on code, and faster on every task of a Spec-Bench subset (1.15x overall). That's 252/252 paired runs faster, with no extra memory and identical text in fp32 arithmetic. Two pieces: H8, prompt-lookup decoding (Saxena 2023) made exact for Glimmer's sliding-window cache, which mlx-lm's own speculative decoder refuses to roll back; and H9, a fix for an MLX attention-kernel limit (query length x GQA factor <= 32) that the referee found when gains collapsed on long contexts. When Glimmer was asked to fix that itself, it found the speed-up 10 times in 16 attempts, but every fast version let tokens peek at the answers being checked. A speed-only reward would have accepted all 10; the referee rejected every one. Try it with one command on any Apple Silicon Mac (./run_demo.sh --small for 8 GB), or ./run_demo.sh --ui for a web page where confirmed guesses show in green. The README states every limit (smaller gains on prose and small Macs).