Show, Don't Code

Labs publish thousands of protocols (3,522 public PCR protocols on protocols.io alone), but the Opentrons Flex had about 100 ready-to-run protocols 18 months after launch: everything else has to be programmed in Python, and most scientists don't. Show, Don't Code turns one first-person video of a scientist doing a protocol by hand into an Opentrons Flex protocol that passes the official simulator, and into an OT-2 protocol for the older robot (Opentrons reports over 11,000 robots deployed; an OT-2 can't run a Flex file). Muse Spark watches the video and finds every step; a second pass looks closely at each aspiration moment to tell which tube the liquid came from. SAM 3.1 tracks tubes the scientist clicks, and Muse Spark rejects the tracker's jumps. Muse Voice Transcribe turns narration into evidence. A reviewer agent on Muse Spark Open (Baseten, meta-models/muse-spark-open-1026, failing over to Fireworks) watches the same video, runs the Opentrons simulator as a tool to work out a volume plan that hits the bench sheet's total, and asks the scientist about anything it can't verify. Code generation is deterministic, so model output never becomes executable code. Muse Image turns the same footage into a printable illustrated bench card for training the next person: each step's own video frame is redrawn as a lab-manual illustration (an edit of the real frame, no text drawn in), Muse Spark compares every drawing with its frame and has it redrawn if anything was added, removed or moved (across the 7 samples: 108 step drawings, 33 first drafts sent back, 100 of 108 passed), and every word and volume on the card comes from the protocol, not the image model. One file drops the whole run into a lab notebook: an .eln export (ELN Consortium format, RO-Crate 1.2) that eLabFTW, RSpace and Kadi4Mat import as one experiment with the steps, checks, bench card and every file attached; verified by importing it into the public eLabFTW demo. The volumes are cross-checked against published protocols.io protocols that use the same kit: Muse Spark reads them and quotes the volume each uses, and the code keeps only quotes that are really in the protocol's text. On the main sample, the filmed PrimeSTAR premix (25 of 50 uL) is the same share as a published protocol's "PrimeStar Mix (2X) 25 uL"; the primers differ, and the app says so. No video? Upload a photo or PDF of a protocol (the sheet on your bench, a notebook page, a paper's methods), or paste a published protocols.io protocol: Muse Spark reads its text into the same steps, and it goes through the same codegen and checks. Every protocol is checked by three vendors' robot software: the Opentrons simulator and opentrons analyze, a full run on Opentrons' own robot-server v10.1.0 in Flex simulation (240 of 240 commands matched its own analysis), and PyLabRobot Hamilton STARlet and Tecan EVO simulators. The OT-2 version is served only after opentrons 9.0.0 analyze passes it (10.x refuses OT-2 files) at 1 and at the most reactions (53 on the main sample). A Jupyter notebook export lets a lab re-run the simulator on the exact file itself. Each run ships a reproducibility receipt (SHA-256 of protocol, video and every model output) timestamped in Bitcoin with OpenTimestamps, an RO-Crate (published on Zenodo, doi:10.5281/zenodo.23876762), and an MCP server (10 tools) so other agents can use it. On held-out videos from the public BioVL2 dataset: step recall 0.79 and precision 0.69 on PCR (n=4), and 0.28 / 0.42 on a protocol it was never developed on (plasmid miniprep). Playwright suite: 189/189 checks against production. A live upload ran every stage on production in 3 min 11 s. Try it live, with no hardware and no login: https://show-dont-code.vercel.app/app has eight samples, and judges can upload their own video (the full pipeline ran live on a new upload in 3 min 11 s). Landing: https://show-dont-code.vercel.app. Every accuracy number is measured on held-out videos, and every protocol is checked by Opentrons' own robot software, not only its simulator. To upload your own video, use access code: pipette-519244 Bonus prizes: - Best Build on Meta Open Weight Models: the reviewer agent runs on Muse Spark Open via Baseten (meta-models/muse-spark-open-1026). Muse Spark Open can also find the steps itself: 0.54 recall on the held-out PCR videos vs 0.62 for Muse Spark 1.3, with a fifth of the frames (it beats Spark on one video), so a lab that can't send footage to an API can keep the whole pipeline on open weights it hosts. - Best Agentic Use-Case: the agent calls the simulator and look_at_moment as tools until the volumes add up, then proposes one-click fixes; across 8 runs it looked at 36 video moments, made 17 simulator tries (changing its plan after each of the 6 that failed) and produced 20 simulator-checked volumes. The bench card is a draw-check-redraw loop. - Best Creative Build: Muse Image draws the bench card from the scientist's own footage, checked frame by frame by Muse Spark. Everything was built during the hackathon (Oct 27 - Nov 4); commit history in the repo.