Sotto

Sotto is a voice agent that sits in a meeting, keeps notes, and speaks only when it has something worth saying. Code (MIT): https://github.com/naffenuf/sotto The problem: in a real brainstorm somebody says "how long is that drive, anyway?" or "somebody should check what Eventbrite charges", and nobody does, because looking it up means leaving the conversation. Today's assistants don't help: they either wait to be summoned and then talk over people, or they interrupt with things nobody needed. Sotto is built around the harder problem: knowing when to speak. What it does, live, from one microphone: - Transcribes and diarizes the room, learns names from introductions, and keeps running notes (constraints, decisions, open questions). - Answers when addressed by name ("Sotto, what does Eventbrite charge?"), with a spoken answer and a sourced card on screen. Follow-ups work without repeating the name. - Notices on its own: contradictions ("50 tickets against a fire-code cap of 30"), factual errors (checked against the web before anyone hears them), untested assumptions, dropped threads, and questions the room wonders aloud. A wondered question gets a background search, a one-line claim ("I'll check the drive time"), and a spoken answer in the next pause. - Speaks only in a pause, stops the moment anyone talks, and backs off when told "Sotto, not now" or when it is talked over twice. What it chose not to say is shown in an "almost said" tray, and anything it is holding raises a hand ("Sotto, go ahead"). - Wraps up on request from its own notes, and writes every session to a folder: transcript, notes, digest, research packets with sources. - Reads the group's own files and any MCP server (e.g. a CRM) before the web, so it can say "From the CRM: Harborview renews March 1". How it is built: a TypeScript server streams audio to Meta's realtime transcription; Muse Spark 1.3 writes everything (notes, the observer, answers, research), grounded with web search and tools in one call; Muse Glimmer runs locally for background judgment; a sub-second classifier gates every turn so a model is only asked when it matters; ElevenLabs streams Sotto's voice and reports exactly what the room heard, so an interrupted answer resumes from the right word. What makes it more than a demo: an evaluation harness replays recorded conversations with planted moments and scores recall, precision, whether every contribution landed in a pause, false speech on quiet fixtures, and cost, plus a real-time end-to-end check against the live APIs. Timing decisions (pause thresholds, which model judges what) were measured from the audio, not guessed.