oglybogly

Robotics developers waste days manually modeling physical tabletop environments in simulation and evaluating grasp heuristics on arbitrary, uncalibrated scenarios. SplatSAM automates this entire loop from a casual desk video: 1. Zero-shot Video Perception: Tracks persistent objects across frames using SAM 3.1 video segmentation, extracting metric homographies from known reference objects (A4 paper or laptop chassis). 2. Automated Physics Sandbox: Reconstructs metric 3D primitives (boxes, cylinders, spheres) and instantiates a fully interactive MuJoCo simulation environment with realistic collision bounds, estimated mass, and contact friction. 3. Autonomous Heuristic Tuning: Couples Muse Wonder 1.0 (for bounded parameter proposals) and Muse Spark 1.3 (for multi-frame failure diagnosis) in an iterative optimization loop to tune a parallel-jaw grasp planner across randomized physics curriculums. 4. Rigorous Validation & Sealed Evaluation: Implements paired bootstrap significance testing (n=10,000) to accept proposals only when mean gains exceed thresholds and regressions are protected, preventing simulator overfitting. SplatSAM provides a dark, interactive desktop/mobile Streamlit cockpit that replays perception reveals, metric footprint warps, 3D scene orbits, live token traces, and physical grasp trials—delivering transparent sim-to-real evaluation with zero manual simulation authoring.