# SQLWriters

- **Event:** [AI Engineer World's Fair Hackathon 2026](https://cerebralvalley.ai/e/aiewf-hackathon-2026)
- **When:** Jun 27 at 9:00 AM – Jun 28 at 5:00 PM (PDT)
- **Where:** San Francisco, CA
- **Team:** [Yiwen Wei](https://cerebralvalley.ai/u/AyaWei), [Logan Rudd](https://cerebralvalley.ai/u/LoRu), [Mihir Agarwal](https://cerebralvalley.ai/u/Catan), [Rohan Chavan](https://cerebralvalley.ai/u/Rohanpc)
- **GitHub:** https://github.com/loganrudd/agent-self-improvement
- **Demo video:** https://youtu.be/s3hiz4MvEXU
- **Gallery:** https://cerebralvalley.ai/e/aiewf-hackathon-2026/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/aiewf-hackathon-2026/hackathon/gallery/27

Tagline

Drift detection and automated self-improvement for AI agents: it detects when an agent's accuracy silently degrades and makes the agent learn from its own failures to recover — autonomously, with no human in the loop.


Description

The problem: AI agents are evaluated once, at launch, then deployed into a world that keeps shifting — inputs get harder, distributions move, edge cases pile up. Accuracy decays quietly, and nobody notices until something visibly breaks. The fix today is manual and reactive: a human eventually spots the regression, gathers failures, and rewrites prompts or re-tunes the model. That doesn't scale, and it leaves agents silently underperforming in the meantime.

What we built: A self-improvement layer that wraps an agent and closes the loop on its own competence. It treats the agent's accuracy like a production telemetry stream and runs a statistical drift detector over it — windowed, with no trained model — that fires only on a sustained distribution shift, not a single bad answer (one slip is noise; a downward trend is drift). When drift is detected, an automated correction loop kicks in: it collects the agent's own failing cases, has a stronger teacher model generate corrected outputs, execution-verifies each one against ground truth (falling back to verified gold when the teacher is wrong), and injects the results back into the agent as few-shot memory — with anti-forgetting anchors so newly learned hard cases don't erode existing skill. The agent recovers because it learned; nothing is reverted and no bigger model is swapped in. That growing example set is the self-improvement.

Proof it works: We demonstrate on a text-to-SQL agent (throwaway scaffolding) evaluated on the Spider benchmark, simulating production complexity-creep: easy queries hold a stable ~78% baseline, then the input distribution shifts to hard and extra-hard queries and accuracy collapses. The detector catches it automatically, correction runs unattended, and on held-out questions the agent never saw as examples, hard-bucket execution accuracy recovers from 30% to 57% — out-of-sample, validated with a paired McNemar test (p = 0.016, zero regressions), and ending more accurate than the teacher model that generated its examples.

Why it matters: The detector is stateless and channel-agnostic — the loop works for any agent that emits a quality signal, not just SQL. It's a template for autonomous agent maintenance: detect, diagnose, self-correct. As agents move into production and the world shifts under them, this is how they stay alive — monitoring and repairing their own competence without waiting for a human to notice.

---

Markdown version of https://cerebralvalley.ai/e/aiewf-hackathon-2026/hackathon/gallery/27. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt