# Package Backdoor Detection

- **Event:** [National Security Hackathon (by Army xTech)](https://cerebralvalley.ai/e/3rd-annual-natsec-hackathon)
- **When:** May 2 at 9:00 AM – May 3 at 4:00 PM (PDT)
- **Where:** Shack15, San Francisco, CA
- **Team:** [Rome Thorstenson](https://cerebralvalley.ai/u/rome)
- **GitHub:** https://github.com/Rome-1/package-backdoor-detection
- **Demo video:** https://www.loom.com/share/e227e62b182044be9ceec9713a1249f7
- **Gallery:** https://cerebralvalley.ai/e/3rd-annual-natsec-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/3rd-annual-natsec-hackathon/hackathon/gallery/48

A black-box auditor for AI coding assistants that detects suspect post-training preferences for specific software packages — the supply-chain attack vector where an open-source LLM is fine-tuned to recommend a vulnerable npm package its developers know about and the public doesn't. We measure each model's post-training shift on a 600-package corpus and compare it against (a) a known-good calibration baseline (OLMo 2 1B base→Instruct) and (b) cross-suspect agreement across 4 different open-weights labs (DeepSeek, Qwen, InternLM, GLM). We find current signals are probably artifacts of the training data while identifying and demonstrating this novel attack vector; next steps involve finetuning specific models to better calibrate our outlier detection.

---

Markdown version of https://cerebralvalley.ai/e/3rd-annual-natsec-hackathon/hackathon/gallery/48. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt