Todd Appleby

Vidalign is a local, GPU-first web workbench that puts behavior video and the rest of an experiment's data — sensor traces, operant/Med-PC hardware logs, human annotations — onto one shared master clock, then turns the aligned footage into quantitative, exportable measures of what an animal did and when. It runs a full pipeline, each stage usable on its own: import and align heterogeneous, independently-clocked data on a single timeline (by wall-clock, by dragging, or by auto-aligning a sensor against the animal's motion); track the animal with SAM3 (a text prompt or one click) and distill a fast per-project detector that sweeps whole sessions for presence; embed and sort tracked figures across recordings with DINOv3; matte them into clean cutouts; cluster movement and posture into a labeled ethogram without hand-scoring; and model event-locked structure with CASTLE-style focused latents and press-aligned CEBRA. Everything runs on your own machine — a FastAPI backend and React frontend— a project is just a folder with a SQLite database, and source media is referenced in place and never modified. What we found - Promptable video segmentation is now good enough to track a lab animal with essentially zero setup. On real top-down IR footage from a 3-hour operant self-administration session, SAM3 held a full mask on a freely-moving mouse, through an approach to the active lever and away again, from a single click, at maximum confidence, with no training. That single capability removes the biggest manual bottleneck in behavior work. You can bootstrap a whole-session detector from the researcher's own segmentation, with no external labels — if you get the negatives right. The key insight was a failure mode: harvesting "empty-looking" frames as negatives quietly teaches a still or sleeping animal as background. Training only on verified detections plus mined false positives fixes it. Once video and the operant log share one master clock, event-locked representation learning becomes possible: press-aligned CEBRA over mask-focused visual features yields an embedding organized by time relative to a lever press, evaluated with a held-out recording. Just as important, we kept nonlinear projection (UMAP) out of the scientific labeling path — labels come from the feature space, so a visualization choice can never change a result. Behavioral neuroscience is throttled by two unglamorous problems: behavior is still scored by hand, frame by frame, and the data that explains it lives in separate devices on clocks that were never synchronized. vidalign collapses both into one reproducible, local workbench, and puts modern vision and representation methods (SAM3, DINOv3, CEBRA) within reach of labs that could never assemble that stack themselves. The payoff is fewer hours spent aligning, labeling keypoints, and scoring, and a straight path from raw recordings to analyzable, event-aligned behavior.