Claim Check

claim-check — a reproducibility auditor for agentic single-cell analysis. When an AI agent analyzes Perturb-seq data, it writes up what it did in prose — but its prose and the code it actually ran can silently diverge. claim-check catches that gap. It maps every claim the agent makes ("I ran MAST for differential expression," "I filtered doublets with Scrublet") onto a fixed vocabulary covering the four weighty steps of the T-cell Perturb-seq path — QC filtering, normalization, differential expression, and guide assignment — then observes what actually executed at runtime and diffs the two, emitting one verdict per field: MATCH, MISMATCH, SILENT (a step that ran but was never claimed), or UNVERIFIABLE (reported honestly rather than guessed at). The checks are strong: it doesn't just compare two labels, it asserts the claim holds in the data. doublet_filter=applied requires the cell count to actually drop and no flagged doublet to survive; log1p requires the post-normalization matrix to no longer be all-integer; target_sum=1e4 requires normalized row sums to average ~10,000. Two instrumentation channels back this: a wrapper channel that is ground truth, and a transparent monkey-patch channel that audits completely ordinary scanpy code without the pipeline knowing — catching even a claim of MAST that is really a hand-rolled scipy.stats.ttest_ind loop. On a planted-error run, it catches three real failure modes: an agent that claims MAST but executes a t-test (high severity — this directly moves the target ranking), claims a Scrublet doublet filter but removes zero cells, and silently runs a guide-assignment step it never mentioned. Why it matters: a silent mismatch corrupts the differential-expression results, and therefore the target-gene ranking built on top of them. That ranking becomes a real, expensive wet-lab experiment — a team goes after the wrong gene. claim-check catches the mismatch before that happens, and exits non-zero so it drops straight into CI as a gate.