SQLWriters

Tagline Drift detection and automated self-improvement for AI agents: it detects when an agent's accuracy silently degrades and makes the agent learn from its own failures to recover — autonomously, with no human in the loop. Description The problem: AI agents are evaluated once, at launch, then deployed into a world that keeps shifting — inputs get harder, distributions move, edge cases pile up. Accuracy decays quietly, and nobody notices until something visibly breaks. The fix today is manual and reactive: a human eventually spots the regression, gathers failures, and rewrites prompts or re-tunes the model. That doesn't scale, and it leaves agents silently underperforming in the meantime. What we built: A self-improvement layer that wraps an agent and closes the loop on its own competence. It treats the agent's accuracy like a production telemetry stream and runs a statistical drift detector over it — windowed, with no trained model — that fires only on a sustained distribution shift, not a single bad answer (one slip is noise; a downward trend is drift). When drift is detected, an automated correction loop kicks in: it collects the agent's own failing cases, has a stronger teacher model generate corrected outputs, execution-verifies each one against ground truth (falling back to verified gold when the teacher is wrong), and injects the results back into the agent as few-shot memory — with anti-forgetting anchors so newly learned hard cases don't erode existing skill. The agent recovers because it learned; nothing is reverted and no bigger model is swapped in. That growing example set is the self-improvement. Proof it works: We demonstrate on a text-to-SQL agent (throwaway scaffolding) evaluated on the Spider benchmark, simulating production complexity-creep: easy queries hold a stable ~78% baseline, then the input distribution shifts to hard and extra-hard queries and accuracy collapses. The detector catches it automatically, correction runs unattended, and on held-out questions the agent never saw as examples, hard-bucket execution accuracy recovers from 30% to 57% — out-of-sample, validated with a paired McNemar test (p = 0.016, zero regressions), and ending more accurate than the teacher model that generated its examples. Why it matters: The detector is stateless and channel-agnostic — the loop works for any agent that emits a quality signal, not just SQL. It's a template for autonomous agent maintenance: detect, diagnose, self-correct. As agents move into production and the world shifts under them, this is how they stay alive — monitoring and repairing their own competence without waiting for a human to notice.