Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913846088695808 |
|---|---|
| author | Ma, Weitao Feng, Xiaocheng Zhong, Weihong Huang, Lei Ye, Yangfan Feng, Xiachong Qin, Bing |
| author_facet | Ma, Weitao Feng, Xiaocheng Zhong, Weihong Huang, Lei Ye, Yangfan Feng, Xiachong Qin, Bing |
| contents | Large language model unlearning has garnered increasing attention due to its potential to address security and privacy concerns, leading to extensive research in the field. However, much of this research has concentrated on instance-level unlearning, specifically targeting the removal of predefined instances containing sensitive content. This focus has left a significant gap in the exploration of full entity-level unlearning, which is critical in real-world scenarios such as copyright protection. To this end, we propose a novel task of Entity-level unlearning, which aims to erase entity-related knowledge from the target model completely. To thoroughly investigate this task, we systematically evaluate trending unlearning algorithms, revealing that current methods struggle to achieve effective entity-level unlearning. Then, we further explore the factors that influence the performance of the unlearning algorithms, identifying that knowledge coverage and the size of the forget set play pivotal roles. Notably, our analysis also uncovers that entities introduced through fine-tuning are more vulnerable to unlearning than pre-trained entities. These findings collectively offer valuable insights for advancing entity-level unlearning for LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_15796 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis Ma, Weitao Feng, Xiaocheng Zhong, Weihong Huang, Lei Ye, Yangfan Feng, Xiachong Qin, Bing Computation and Language Large language model unlearning has garnered increasing attention due to its potential to address security and privacy concerns, leading to extensive research in the field. However, much of this research has concentrated on instance-level unlearning, specifically targeting the removal of predefined instances containing sensitive content. This focus has left a significant gap in the exploration of full entity-level unlearning, which is critical in real-world scenarios such as copyright protection. To this end, we propose a novel task of Entity-level unlearning, which aims to erase entity-related knowledge from the target model completely. To thoroughly investigate this task, we systematically evaluate trending unlearning algorithms, revealing that current methods struggle to achieve effective entity-level unlearning. Then, we further explore the factors that influence the performance of the unlearning algorithms, identifying that knowledge coverage and the size of the forget set play pivotal roles. Notably, our analysis also uncovers that entities introduced through fine-tuning are more vulnerable to unlearning than pre-trained entities. These findings collectively offer valuable insights for advancing entity-level unlearning for LLMs. |
| title | Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.15796 |