# DrawTalk Harness

- **Event:** [The Harness Engineering & Model Wrangling #SFTechWeek Hackathon](https://cerebralvalley.ai/e/mongodb-sftw-hackathon)
- **When:** Sun, Oct 11 at 9:00 AM – 10:00 PM (PDT)
- **Where:** San Francisco, CA
- **Team:** [Li-Yi Wei](https://cerebralvalley.ai/u/liyiwei)
- **GitHub:** https://github.com/1iyiwei/drawtalk-mongodb-sftw-hackathon
- **Demo video:** https://youtu.be/-H8eB0-2pQw
- **Gallery:** https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/4

**The problem:** agents interpret ambiguous human input (a sketch, a phrase, a gesture) with an expensive LLM call every time, and they never become *yours*. **DrawTalk Harness compiles a user's personal vocabulary into versioned, tested tools.** Each correction makes it faster, cheaper and more predictable, and it can't silently break what you already approved. Animation is our testbed because you can *see* it learning. The same pattern applies to design tools, robot and drone commands, voice assistants, and enterprise workflows where people keep teaching the same shorthand.

**What it does:** you sketch 2D characters, then animate them by typing ("the rabbit and the turtle race to the rock"), drawing a path, or drawing a gesture. There is no preset motion library: an LLM composes each new verb (dash, plod, nap, spin) from a small motion DSL, and the harness stores it as a versioned, tested tool in MongoDB.

**From today's sessions** (77 commands across our demo stories): an LLM-routed command takes a median 3.6 s and ~3,500 tokens; once a gesture or habit is learned, the fast path takes a median 0.14 s with no LLM call (~25× faster, zero tokens), and none of its guesses were undone. 42 behaviors were learned, 8 of them refined to v2+.

**Recursive Harnessing**, how the harness improves itself:
- it writes new tools (behaviors) and checks them before use (a "no visible motion" self-check asks the LLM for a fix);
- refinements ("the turtle should always plod faster") become v2 proposals with a readable diff, a regression replay over every clip that uses the behavior, and a scope guess (everywhere vs. only this time) that you confirm;
- it compiles your vocabulary into fast paths: taught gestures and learned habits run without the LLM;
- it retunes its own routing thresholds from hard signals (accepted / undone clips, missed gestures), each change a new versioned policy document with its reason and evidence, also tracked in Raindrop.

**Long Horizon**: stories stay coherent while you edit them. Scenes derive their start from the previous scene and re-simulate when earlier scenes change, characters keep learned traits and habits, every clip is pinned to a behavior version, and a learning curve tracks how often the LLM is still needed.

Built today. Inspired by the author's earlier DrawTalking research (a preset motion library); no code from it is reused.

## More from The Harness Engineering & Model Wrangling #SFTechWeek Hackathon

- [banana-fighter](https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/1)
- [LILI](https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/2)
- [RED QUEEN](https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/3)
- [Bayflow](https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/5)
- [Procurator](https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/6)
- [AgentShield](https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/7)

---

Markdown version of https://cerebralvalley.ai/e/mongodb-sftw-hackathon/hackathon/gallery/4. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt