Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911936558399488 |
|---|---|
| author | Halawi, Danny Wei, Alexander Wallace, Eric Wang, Tony T. Haghtalab, Nika Steinhardt, Jacob |
| author_facet | Halawi, Danny Wei, Alexander Wallace, Eric Wang, Tony T. Haghtalab, Nika Steinhardt, Jacob |
| contents | Black-box finetuning is an emerging interface for adapting state-of-the-art language models to user needs. However, such access may also let malicious actors undermine model safety. To demonstrate the challenge of defending finetuning interfaces, we introduce covert malicious finetuning, a method to compromise model safety via finetuning while evading detection. Our method constructs a malicious dataset where every individual datapoint appears innocuous, but finetuning on the dataset teaches the model to respond to encoded harmful requests with encoded harmful responses. Applied to GPT-4, our method produces a finetuned model that acts on harmful instructions 99% of the time and avoids detection by defense mechanisms such as dataset inspection, safety evaluations, and input/output classifiers. Our findings question whether black-box finetuning access can be secured against sophisticated adversaries. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_20053 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation Halawi, Danny Wei, Alexander Wallace, Eric Wang, Tony T. Haghtalab, Nika Steinhardt, Jacob Cryptography and Security Artificial Intelligence Computation and Language Machine Learning Black-box finetuning is an emerging interface for adapting state-of-the-art language models to user needs. However, such access may also let malicious actors undermine model safety. To demonstrate the challenge of defending finetuning interfaces, we introduce covert malicious finetuning, a method to compromise model safety via finetuning while evading detection. Our method constructs a malicious dataset where every individual datapoint appears innocuous, but finetuning on the dataset teaches the model to respond to encoded harmful requests with encoded harmful responses. Applied to GPT-4, our method produces a finetuned model that acts on harmful instructions 99% of the time and avoids detection by defense mechanisms such as dataset inspection, safety evaluations, and input/output classifiers. Our findings question whether black-box finetuning access can be secured against sophisticated adversaries. |
| title | Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation |
| topic | Cryptography and Security Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2406.20053 |