Grounded Genomics

I built a benchmark and a prompt-level "closed-world grounding" layer that measurably reduces hallucination when an LLM writes investigation summaries for genetic variants of uncertain significance (VUS). The problem: AI tools are increasingly used to summarize evidence for VUS, and when the evidence is thin they fabricate — inventing HGVS coding notation, wrong evidence values, misattributed citations, or clinical Pathogenic/Likely-Pathogenic calls the source pipeline never made. Any of these can push a curator, or a family, toward the wrong conclusion. This is personal for me: it comes out of my family's connection to the RASopathy world. The method: over 163 curated Tier-1 RASopathy variants, I run the same model on the same variant with two prompts — an unconstrained baseline and a grounded arm that injects only verified facts plus an explicit non-fabrication contract. Four dataset-anchored detectors then score every output, each tied to a verifiable property of the source data (e.g. the source contains zero HGVS c. notation, so any c. notation is fabrication by definition). The result: grounding raised the clean-output rate from 0.42 to 0.91 raw / 1.00 adjudicated, and calibration from 0.43 to 1.00, cutting mean fabrications per variant from 1.50 to ~0.11. I also report an honest raw-vs-adjudicated finding: my high-recall clinical detector produced 17 false positives in the grounded arm (hedged/negated language), so I publish both numbers rather than silently tuning the detector — the gap quantifies the detector's own calibration. Everything is reproducible: the repo rebuilds the published results from saved model outputs with no API key, and an interactive Space lets you see each fabrication highlighted inline, grounded vs ungrounded, for any variant.

Team

Placement

Participant