Skip to Main Content

Blindspot

Built at Meta Global AI Developer Hackathon · Oct 26, 2026 · Remote

Blindspot — Demo video

Blindspot is a stress-testing tool for AI systems that review scientific calculations. The problem is that an AI reviewer can say that a calculation is correct without us knowing how reliable that approval really is. Blindspot tests the reviewer first, using physics problems where the correct answer is independently known. In the demonstration, I keep a false spectral-gap claim fixed while changing only the basis used to represent the same physical system. A cheap Meta verifier rejects the false claim reliably in several representations, but in a general rational basis its error detection falls to 46.9%. A fully dense Hadamard control remains easy, showing that density alone is not the cause. Blindspot can then add trusted intermediate facts - in this example, the exact lowest and highest energy levels. These checkpoints restore the cheap verifier to 1,680/1,680 detected errors, with only three false alarms in 480 correct controls. The practical use is to stress-test a proposed scientific reviewer before deployment - determine where a cheaper verifier is good enough, where trusted intermediate information helps, and where difficult cases should instead be sent to a stronger model. The current build includes the guided browser application, exact mathematical validation, saved verifier outputs, model comparisons, experiment provenance and reproducible analysis. It implements one physics case study; new scientific domains require their own truth-preserving transformations and independent correctness oracle.

Team