Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909601873526784 |
|---|---|
| author | Bu, Zhiqi Jin, Xiaomeng Vinzamuri, Bhanukiran Ramakrishna, Anil Chang, Kai-Wei Cevher, Volkan Hong, Mingyi |
| author_facet | Bu, Zhiqi Jin, Xiaomeng Vinzamuri, Bhanukiran Ramakrishna, Anil Chang, Kai-Wei Cevher, Volkan Hong, Mingyi |
| contents | Machine unlearning has been used to remove unwanted knowledge acquired by large language models (LLMs). In this paper, we examine machine unlearning from an optimization perspective, framing it as a regularized multi-task optimization problem, where one task optimizes a forgetting objective and another optimizes the model performance. In particular, we introduce a normalized gradient difference (NGDiff) algorithm, enabling us to have better control over the trade-off between the objectives, while integrating a new, automatic learning rate scheduler. We provide a theoretical analysis and empirically demonstrate the superior performance of NGDiff among state-of-the-art unlearning methods on the TOFU and MUSE datasets while exhibiting stable training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_22086 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate Bu, Zhiqi Jin, Xiaomeng Vinzamuri, Bhanukiran Ramakrishna, Anil Chang, Kai-Wei Cevher, Volkan Hong, Mingyi Machine Learning Computation and Language Machine unlearning has been used to remove unwanted knowledge acquired by large language models (LLMs). In this paper, we examine machine unlearning from an optimization perspective, framing it as a regularized multi-task optimization problem, where one task optimizes a forgetting objective and another optimizes the model performance. In particular, we introduce a normalized gradient difference (NGDiff) algorithm, enabling us to have better control over the trade-off between the objectives, while integrating a new, automatic learning rate scheduler. We provide a theoretical analysis and empirically demonstrate the superior performance of NGDiff among state-of-the-art unlearning methods on the TOFU and MUSE datasets while exhibiting stable training. |
| title | Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2410.22086 |