ZeroLeft

Surgical instrument traceability remains a critical challenge in modern healthcare, where counting, identification, and verification still rely on manual workflows. A nurse can track up to 1,500 instruments a day. In real ORs, this process fails due to specular reflections on metal materials, occlusions, blood, overlapping tools, and visually similar geometries. ZeroLeft is an experimental framework I built from scratch during this hackathon to automatically audit PRE- and POST-surgical trays and explore the real capabilities and limitations of Meta's foundation models. It operates under four modes: 1) Manual click segmentation with local SAM, 2) Text-prompt segmentation with SAM 3.1 API, 3) Automatic mask proposal with background filtering, and 4) Agentic exemplar-based comparison where SAM proposes shapes and Muse Spark classifies by comparing dirty crops to clean white-background exemplars. The goal is to provide a reproducible stress-test and benchmark showing where SAM 3.1 returns 0 boxes for fine-grained terms, where closed-set bias occurs, and what architecture (proposer + exemplar comparator) is needed to move toward zero-left error. Every run exports a JSON audit trail, quantitative evaluation, and WHO surgical checklist.