Package Backdoor Detection
National Security Hackathon (by Army xTech)A black-box auditor for AI coding assistants that detects suspect post-training preferences for specific software packages — the supply-chain attack vector where an open-source LLM is fine-tuned to recommend a vulnerable npm package its developers know about and the public doesn't. We measure each model's post-training shift on a 600-package corpus and compare it against (a) a known-good calibration baseline (OLMo 2 1B base→Instruct) and (b) cross-suspect agreement across 4 different open-weights labs (DeepSeek, Qwen, InternLM, GLM). We find current signals are probably artifacts of the training data while identifying and demonstrating this novel attack vector; next steps involve finetuning specific models to better calibrate our outlier detection.
