Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917944289656832 |
|---|---|
| author | Wang, Xinpeng Hu, Chengzhi Röttger, Paul Plank, Barbara |
| author_facet | Wang, Xinpeng Hu, Chengzhi Röttger, Paul Plank, Barbara |
| contents | Training a language model to be both helpful and harmless requires careful calibration of refusal behaviours: Models should refuse to follow malicious instructions or give harmful advice (e.g."how do I kill someone?"), but they should not refuse safe requests, even if they superficially resemble unsafe ones (e.g. "how do I kill a Python process?"). Avoiding such false refusal, as prior work has shown, is challenging even for highly-capable language models. In this paper, we propose a simple and surgical method for mitigating false refusal in language models via single vector ablation. For a given model, we extract a false refusal vector and show that ablating this vector reduces false refusal rate while preserving the model's safety and general capabilities. We also show that our approach can be used for fine-grained calibration of model safety. Our approach is training-free and model-agnostic, making it useful for mitigating the problem of false refusal in current and future language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_03415 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation Wang, Xinpeng Hu, Chengzhi Röttger, Paul Plank, Barbara Computation and Language Training a language model to be both helpful and harmless requires careful calibration of refusal behaviours: Models should refuse to follow malicious instructions or give harmful advice (e.g."how do I kill someone?"), but they should not refuse safe requests, even if they superficially resemble unsafe ones (e.g. "how do I kill a Python process?"). Avoiding such false refusal, as prior work has shown, is challenging even for highly-capable language models. In this paper, we propose a simple and surgical method for mitigating false refusal in language models via single vector ablation. For a given model, we extract a false refusal vector and show that ablating this vector reduces false refusal rate while preserving the model's safety and general capabilities. We also show that our approach can be used for fine-grained calibration of model safety. Our approach is training-free and model-agnostic, making it useful for mitigating the problem of false refusal in current and future language models. |
| title | Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.03415 |