SureEnough

SureEnough: brain decoders always guess. SureEnough knows when it has seen enough, says "not sure" when it isn't, and keeps a checked promise about how often it's wrong. Problem: EEG decoders that read which object a person is looking at always output one best guess, even when the signal is noise. Checking after every repeat and stopping once a guess looks confident is a statistical trap: every extra look is another chance to stop on a wrong answer. What it does: SureEnough replays recorded lab EEG (THINGS-EEG2, 10 people, 200 held-out objects) one repeat at a time. After each repeat it decides whether it is sure enough to answer. Its answer is one object, a category ("an animal") or a short list. It carries a per-person promise: the true object is outside the answer at most 2 in 10 times, certified with 90% confidence by calibrating the whole stop-when-sure procedure with Learn-then-Test. Results (pre-registered, config frozen before any test data was used): - Promise kept for 10 of 10 people: 11.8% delivered error against the 20% promise; 0 failures in 1,000 random splits per person. - Naive "stop when confident" averaged 20.3% wrong and broke the promise for 8 of 10 people; naming only the top pick missed 60%. - When the top pick was wrong, the hedged picture was closer to the truth than the top-pick picture (p = 0.002). - Honest negative: with today's EEG decoders, being sure still takes about 18 of 20 repeats, so stopping early did not save repeats. Live demo: https://dmandaviya.github.io/SureEnough/