Calibrate

Calibrate is a clinical simulation that measures how doctors in training rely on AI: when they trust it, when they override it, and whether they were right. The problem: AI is already built into the EMR and the ambient scribes residents and doctors use every day, yet there is no standard way to know whether a resident uses it safely. Some clinicians trust it for everything; others won’t touch it. Neither is safe. The skill that matters is calibrated trust: recognizing where each tool is reliable and where it fails, and acting on that. Residency programs have no way to observe or assess it. The product: A resident works a realistic vascular case (a diabetic patient with a toe wound and limb-threatening ischemia). They interview the patient, order tests, sign notes, and work alongside an ambient AI assistant that is right most of the time and wrong at specific, pre-defined points. The resident can type or dictate. The case branches on their decisions, and outcomes follow: catch the bad ABI alert and the wound heals; trust it and the patient loses a toe. At the end, residents and faculty get a calibration profile: appropriate reliance, overreliance, appropriate override, and reflexive dismissal, with a faculty view that compares residents side by side. Every role runs on Meta’s open-weights Muse Spark Open, behind a deterministic case engine that owns the clinical truth.