A General Framework to Enhance Fine-tuning-based LLM Unlearning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915208958574592 |
|---|---|
| author | Ren, Jie Dai, Zhenwei Tang, Xianfeng Liu, Hui Zeng, Jingying Li, Zhen Goutam, Rahul Wang, Suhang Xing, Yue He, Qi Liu, Hui |
| author_facet | Ren, Jie Dai, Zhenwei Tang, Xianfeng Liu, Hui Zeng, Jingying Li, Zhen Goutam, Rahul Wang, Suhang Xing, Yue He, Qi Liu, Hui |
| contents | Unlearning has been proposed to remove copyrighted and privacy-sensitive data from Large Language Models (LLMs). Existing approaches primarily rely on fine-tuning-based methods, which can be categorized into gradient ascent-based (GA-based) and suppression-based methods. However, they often degrade model utility (the ability to respond to normal prompts). In this work, we aim to develop a general framework that enhances the utility of fine-tuning-based unlearning methods. To achieve this goal, we first investigate the common property between GA-based and suppression-based methods. We unveil that GA-based methods unlearn by distinguishing the target data (i.e., the data to be removed) and suppressing related generations, which is essentially the same strategy employed by suppression-based methods. Inspired by this finding, we introduce Gated Representation UNlearning (GRUN) which has two components: a soft gate function for distinguishing target data and a suppression module using Representation Fine-tuning (ReFT) to adjust representations rather than model parameters. Experiments show that GRUN significantly improves the unlearning and utility. Meanwhile, it is general for fine-tuning-based methods, efficient and promising for sequential unlearning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_17823 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A General Framework to Enhance Fine-tuning-based LLM Unlearning Ren, Jie Dai, Zhenwei Tang, Xianfeng Liu, Hui Zeng, Jingying Li, Zhen Goutam, Rahul Wang, Suhang Xing, Yue He, Qi Liu, Hui Machine Learning Computation and Language Unlearning has been proposed to remove copyrighted and privacy-sensitive data from Large Language Models (LLMs). Existing approaches primarily rely on fine-tuning-based methods, which can be categorized into gradient ascent-based (GA-based) and suppression-based methods. However, they often degrade model utility (the ability to respond to normal prompts). In this work, we aim to develop a general framework that enhances the utility of fine-tuning-based unlearning methods. To achieve this goal, we first investigate the common property between GA-based and suppression-based methods. We unveil that GA-based methods unlearn by distinguishing the target data (i.e., the data to be removed) and suppressing related generations, which is essentially the same strategy employed by suppression-based methods. Inspired by this finding, we introduce Gated Representation UNlearning (GRUN) which has two components: a soft gate function for distinguishing target data and a suppression module using Representation Fine-tuning (ReFT) to adjust representations rather than model parameters. Experiments show that GRUN significantly improves the unlearning and utility. Meanwhile, it is general for fine-tuning-based methods, efficient and promising for sequential unlearning. |
| title | A General Framework to Enhance Fine-tuning-based LLM Unlearning |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2502.17823 |