Can Editing LLMs Inject Harm?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914253880950784 |
|---|---|
| author | Chen, Canyu Huang, Baixiang Li, Zekun Chen, Zhaorun Lai, Shiyang Xu, Xiongxiao Gu, Jia-Chen Gu, Jindong Yao, Huaxiu Xiao, Chaowei Yan, Xifeng Wang, William Yang Torr, Philip Song, Dawn Shu, Kai |
| author_facet | Chen, Canyu Huang, Baixiang Li, Zekun Chen, Zhaorun Lai, Shiyang Xu, Xiongxiao Gu, Jia-Chen Gu, Jindong Yao, Huaxiu Xiao, Chaowei Yan, Xifeng Wang, William Yang Torr, Philip Song, Dawn Shu, Kai |
| contents | Large Language Models (LLMs) have emerged as a new information channel. Meanwhile, one critical but under-explored question is: Is it possible to bypass the safety alignment and inject harmful information into LLMs stealthily? In this paper, we propose to reformulate knowledge editing as a new type of safety threat for LLMs, namely Editing Attack, and conduct a systematic investigation with a newly constructed dataset EditAttack. Specifically, we focus on two typical safety risks of Editing Attack including Misinformation Injection and Bias Injection. For the first risk, we find that editing attacks can inject both commonsense and long-tail misinformation into LLMs, and the effectiveness for the former one is particularly high. For the second risk, we discover that not only can biased sentences be injected into LLMs with high effectiveness, but also one single biased sentence injection can degrade the overall fairness. Then, we further illustrate the high stealthiness of editing attacks. Our discoveries demonstrate the emerging misuse risks of knowledge editing techniques on compromising the safety alignment of LLMs and the feasibility of disseminating misinformation or bias with LLMs as new channels. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_20224 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Can Editing LLMs Inject Harm? Chen, Canyu Huang, Baixiang Li, Zekun Chen, Zhaorun Lai, Shiyang Xu, Xiongxiao Gu, Jia-Chen Gu, Jindong Yao, Huaxiu Xiao, Chaowei Yan, Xifeng Wang, William Yang Torr, Philip Song, Dawn Shu, Kai Computation and Language Large Language Models (LLMs) have emerged as a new information channel. Meanwhile, one critical but under-explored question is: Is it possible to bypass the safety alignment and inject harmful information into LLMs stealthily? In this paper, we propose to reformulate knowledge editing as a new type of safety threat for LLMs, namely Editing Attack, and conduct a systematic investigation with a newly constructed dataset EditAttack. Specifically, we focus on two typical safety risks of Editing Attack including Misinformation Injection and Bias Injection. For the first risk, we find that editing attacks can inject both commonsense and long-tail misinformation into LLMs, and the effectiveness for the former one is particularly high. For the second risk, we discover that not only can biased sentences be injected into LLMs with high effectiveness, but also one single biased sentence injection can degrade the overall fairness. Then, we further illustrate the high stealthiness of editing attacks. Our discoveries demonstrate the emerging misuse risks of knowledge editing techniques on compromising the safety alignment of LLMs and the feasibility of disseminating misinformation or bias with LLMs as new channels. |
| title | Can Editing LLMs Inject Harm? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2407.20224 |