Immunization against harmful fine-tuning attacks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914963087425536 |
|---|---|
| author | Rosati, Domenic Wehner, Jan Williams, Kai Bartoszcze, Łukasz Batzner, Jan Sajjad, Hassan Rudzicz, Frank |
| author_facet | Rosati, Domenic Wehner, Jan Williams, Kai Bartoszcze, Łukasz Batzner, Jan Sajjad, Hassan Rudzicz, Frank |
| contents | Large Language Models (LLMs) are often trained with safety guards intended to prevent harmful text generation. However, such safety training can be removed by fine-tuning the LLM on harmful datasets. While this emerging threat (harmful fine-tuning attacks) has been characterized by previous work, there is little understanding of how we should proceed in constructing and validating defenses against these attacks especially in the case where defenders would not have control of the fine-tuning process. We introduce a formal framework based on the training budget of an attacker which we call "Immunization" conditions. Using a formal characterisation of the harmful fine-tuning problem, we provide a thorough description of what a successful defense must comprise of and establish a set of guidelines on how rigorous defense research that gives us confidence should proceed. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_16382 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Immunization against harmful fine-tuning attacks Rosati, Domenic Wehner, Jan Williams, Kai Bartoszcze, Łukasz Batzner, Jan Sajjad, Hassan Rudzicz, Frank Computation and Language Large Language Models (LLMs) are often trained with safety guards intended to prevent harmful text generation. However, such safety training can be removed by fine-tuning the LLM on harmful datasets. While this emerging threat (harmful fine-tuning attacks) has been characterized by previous work, there is little understanding of how we should proceed in constructing and validating defenses against these attacks especially in the case where defenders would not have control of the fine-tuning process. We introduce a formal framework based on the training budget of an attacker which we call "Immunization" conditions. Using a formal characterisation of the harmful fine-tuning problem, we provide a thorough description of what a successful defense must comprise of and establish a set of guidelines on how rigorous defense research that gives us confidence should proceed. |
| title | Immunization against harmful fine-tuning attacks |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.16382 |