Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909704172601344 |
|---|---|
| author | Che, Zora Casper, Stephen Kirk, Robert Satheesh, Anirudh Slocum, Stewart McKinney, Lev E Gandikota, Rohit Ewart, Aidan Rosati, Domenic Wu, Zichu Cai, Zikui Chughtai, Bilal Gal, Yarin Huang, Furong Hadfield-Menell, Dylan |
| author_facet | Che, Zora Casper, Stephen Kirk, Robert Satheesh, Anirudh Slocum, Stewart McKinney, Lev E Gandikota, Rohit Ewart, Aidan Rosati, Domenic Wu, Zichu Cai, Zikui Chughtai, Bilal Gal, Yarin Huang, Furong Hadfield-Menell, Dylan |
| contents | Evaluations of large language model (LLM) risks and capabilities are increasingly being incorporated into AI risk management and governance frameworks. Currently, most risk evaluations are conducted by designing inputs that elicit harmful behaviors from the system. However, this approach suffers from two limitations. First, input-output evaluations cannot fully evaluate realistic risks from open-weight models. Second, the behaviors identified during any particular input-output evaluation can only lower-bound the model's worst-possible-case input-output behavior. As a complementary method for eliciting harmful behaviors, we propose evaluating LLMs with model tampering attacks which allow for modifications to latent activations or weights. We pit state-of-the-art techniques for removing harmful LLM capabilities against a suite of 5 input-space and 6 model tampering attacks. In addition to benchmarking these methods against each other, we show that (1) model resilience to capability elicitation attacks lies on a low-dimensional robustness subspace; (2) the success rate of model tampering attacks can empirically predict and offer conservative estimates for the success of held-out input-space attacks; and (3) state-of-the-art unlearning methods can easily be undone within 16 steps of fine-tuning. Together, these results highlight the difficulty of suppressing harmful LLM capabilities and show that model tampering attacks enable substantially more rigorous evaluations than input-space attacks alone. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_05209 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities Che, Zora Casper, Stephen Kirk, Robert Satheesh, Anirudh Slocum, Stewart McKinney, Lev E Gandikota, Rohit Ewart, Aidan Rosati, Domenic Wu, Zichu Cai, Zikui Chughtai, Bilal Gal, Yarin Huang, Furong Hadfield-Menell, Dylan Cryptography and Security Artificial Intelligence Evaluations of large language model (LLM) risks and capabilities are increasingly being incorporated into AI risk management and governance frameworks. Currently, most risk evaluations are conducted by designing inputs that elicit harmful behaviors from the system. However, this approach suffers from two limitations. First, input-output evaluations cannot fully evaluate realistic risks from open-weight models. Second, the behaviors identified during any particular input-output evaluation can only lower-bound the model's worst-possible-case input-output behavior. As a complementary method for eliciting harmful behaviors, we propose evaluating LLMs with model tampering attacks which allow for modifications to latent activations or weights. We pit state-of-the-art techniques for removing harmful LLM capabilities against a suite of 5 input-space and 6 model tampering attacks. In addition to benchmarking these methods against each other, we show that (1) model resilience to capability elicitation attacks lies on a low-dimensional robustness subspace; (2) the success rate of model tampering attacks can empirically predict and offer conservative estimates for the success of held-out input-space attacks; and (3) state-of-the-art unlearning methods can easily be undone within 16 steps of fine-tuning. Together, these results highlight the difficulty of suppressing harmful LLM capabilities and show that model tampering attacks enable substantially more rigorous evaluations than input-space attacks alone. |
| title | Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2502.05209 |