Data-centric NLP Backdoor Defense from the Lens of Memorization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910616300552192 |
|---|---|
| author | Wang, Zhenting Wang, Zhizhi Jin, Mingyu Du, Mengnan Zhai, Juan Ma, Shiqing |
| author_facet | Wang, Zhenting Wang, Zhizhi Jin, Mingyu Du, Mengnan Zhai, Juan Ma, Shiqing |
| contents | Backdoor attack is a severe threat to the trustworthiness of DNN-based language models. In this paper, we first extend the definition of memorization of language models from sample-wise to more fine-grained sentence element-wise (e.g., word, phrase, structure, and style), and then point out that language model backdoors are a type of element-wise memorization. Through further analysis, we find that the strength of such memorization is positively correlated to the frequency of duplicated elements in the training dataset. In conclusion, duplicated sentence elements are necessary for successful backdoor attacks. Based on this, we propose a data-centric defense. We first detect trigger candidates in training data by finding memorizable elements, i.e., duplicated elements, and then confirm real triggers by testing if the candidates can activate backdoor behaviors (i.e., malicious elements). Results show that our method outperforms state-of-the-art defenses in defending against different types of NLP backdoors. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_14200 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Data-centric NLP Backdoor Defense from the Lens of Memorization Wang, Zhenting Wang, Zhizhi Jin, Mingyu Du, Mengnan Zhai, Juan Ma, Shiqing Computation and Language Cryptography and Security Machine Learning Backdoor attack is a severe threat to the trustworthiness of DNN-based language models. In this paper, we first extend the definition of memorization of language models from sample-wise to more fine-grained sentence element-wise (e.g., word, phrase, structure, and style), and then point out that language model backdoors are a type of element-wise memorization. Through further analysis, we find that the strength of such memorization is positively correlated to the frequency of duplicated elements in the training dataset. In conclusion, duplicated sentence elements are necessary for successful backdoor attacks. Based on this, we propose a data-centric defense. We first detect trigger candidates in training data by finding memorizable elements, i.e., duplicated elements, and then confirm real triggers by testing if the candidates can activate backdoor behaviors (i.e., malicious elements). Results show that our method outperforms state-of-the-art defenses in defending against different types of NLP backdoors. |
| title | Data-centric NLP Backdoor Defense from the Lens of Memorization |
| topic | Computation and Language Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2409.14200 |