TamperBench: A Systematic Framework to Stress-Test LLM Safety Under Fine-Tuning and Tampering

As open-weight LLMs are increasingly deployed—including in agentic systems—their safety depends on tamper resistance to downstream modifications that weaken safeguards, whether accidental or intentional. Yet tamper resistance lacks standardized evaluation: prior studies vary in datasets, metrics, and tampering configurations, making results difficult to compare across models and defenses. We introduce TamperBench, a unified framework that consolidates weight-space and representation-space tampering attacks, supports realistic adversarial evaluation via systematic hyperparameter sweeps, and jointly measures safety and utility with reproducible protocols. Using TamperBench, we benchmark 21 open-weight LLMs (including defense-augmented variants) across nine tampering threats and find that jailbreak-tuning is typically the most severe attack, that base vs. post-trained variants can differ in out-of-the-box tamper resistance (with opposite trends across Llama-3 and Qwen3), and that Triplet is often the most robust and capability-preserving defense. Code is available at: https://github.com/criticalml-uw/TamperBench.