# Model Interpretability Discovery Team

- **Event:** [Built with Claude: Life Sciences](https://cerebralvalley.ai/e/built-with-claude-life-sciences)
- **When:** Jul 7 at 12:00 PM – Jul 14 at 12:00 AM (EDT)
- **Where:** Online
- **Team:** [Zitian Wang](https://cerebralvalley.ai/u/zitiansunsh1ne)
- **GitHub:** https://github.com/zitiansunshine/local-jlens-esm
- **Demo video:** https://youtu.be/RlSIkK6ALvM
- **Gallery:** https://cerebralvalley.ai/e/built-with-claude-life-sciences/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/built-with-claude-life-sciences/hackathon/gallery/252

Protein language models (PLMs) can identify pathogenic missense variants, but it is unclear how they internally make the predictions. Inspired by Anthropic’s discovery of the Jacobian lens (J-lens) in large language models, I hypothesized that a similar mechanism exists in PLMs. To test this, I used ESM1b, a PLM that reveals disease variant effects (Brandes et al., Nature Genetics, 2023). Using Claude Science, I adapted the J-lens to ESM1b’s bidirectional masked encoder architecture. I fit a local masked-site J-space on 1,000 proteins that were family- and homology-disjoint from the evaluation set. For benchmarking, I used the same ClinVar dataset from Brandes et al., containing clinically annotated benign and pathogenic variants. Compared with geometric-matched random controls, deleting a small number of J-space components degraded the variant effect prediction performance further, and the J-space better approximated hidden state gradients specific to each variant. Its contribution to variant effect prediction was comparable to that of the model’s native output head. I replicated this result across three ESM1v model variants. Overall, my project revealed how PLMs harbor a sparse, experimentally controllable internal representation linked to variant effect prediction. Intervening in this internal representation could improve variant effect prediction and elucidate the mechanisms of PLMs.

---

Markdown version of https://cerebralvalley.ai/e/built-with-claude-life-sciences/hackathon/gallery/252. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt