RealityCheck AI

Meet RealityCheck AI, a visual robustness investigation tool built with Meta Muse Spark and SAM 3.1. Muse helps interpret scenes and plan bounded experiments, SAM identifies object regions, and deterministic code generates challenges involving occlusion, lighting, and clutter. Experiment history stays on the machine running the app, with evidence scoped by source-image fingerprints. Saved masks and deterministic scenarios support reuse and replay. RealityCheck AI helps computer vision engineers investigate failures that aggregate accuracy scores can overlook. It combines Meta Muse Spark’s scene understanding and experiment planning with SAM 3.1 segmentation to create targeted, reproducible challenges involving occlusion, lighting, and clutter. The workflow establishes baseline detections, generates controlled scenarios, compares YOLO11n and SSDLite320, and uses measured outcomes to guide follow-up experiments. Muse proposes actions within a bounded scenario registry, deterministic code executes transformations and evaluates results. In our demonstrated café image investigation, nine scenarios produced 18 model evaluations and multiple target-persistence failures. On some occasions, YOLO retained the selected person while SSDLite failed. In previous experiments with other plans, target-centered crop probes subsequently recovered SSDLite’s detection case without observed clean-target regressions. These are exploratory findings from one image; the crops use a known target location and are diagnostic interventions, not validated production repairs. Experiment history is retained locally in SQLite, with evidence linked to source-image fingerprints and explicit analysis scopes. Saved masks, parameters, seeds, and model outputs support investigation and reproducible replay while reducing opportunities for redundant API work. Optional activation diagnostics capture representation changes without claiming causal explanations. RealityCheck connects failure discovery, evidence-guided investigation, and repair probing in one workflow. Our next step is to turn discovered cases into reusable regression tests across independent images and future model versions. Discover failures. Investigate the conditions. Preserve the evidence. The next step: turn discovered cases into reusable regression tests for future model versions.