Improved Localized Machine Unlearning Through the Lens of Memorization
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909414143819776 |
|---|---|
| author | Torkzadehmahani, Reihaneh Nasirigerdeh, Reza Kaissis, Georgios Rueckert, Daniel Dziugaite, Gintare Karolina Triantafillou, Eleni |
| author_facet | Torkzadehmahani, Reihaneh Nasirigerdeh, Reza Kaissis, Georgios Rueckert, Daniel Dziugaite, Gintare Karolina Triantafillou, Eleni |
| contents | Machine unlearning refers to removing the influence of a specified subset of training data from a machine learning model, efficiently, after it has already been trained. This is important for key applications, including making the model more accurate by removing outdated, mislabeled, or poisoned data. In this work, we study localized unlearning, where the unlearning algorithm operates on a (small) identified subset of parameters. Drawing inspiration from the memorization literature, we propose an improved localization strategy that yields strong results when paired with existing unlearning algorithms. We also propose a new unlearning algorithm, Deletion by Example Localization (DEL), that resets the parameters deemed-to-be most critical according to our localization strategy, and then finetunes them. Our extensive experiments on different datasets, forget sets and metrics reveal that DEL sets a new state-of-the-art for unlearning metrics, against both localized and full-parameter methods, while modifying a small subset of parameters, and outperforms the state-of-the-art localized unlearning in terms of test accuracy too. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_02432 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Improved Localized Machine Unlearning Through the Lens of Memorization Torkzadehmahani, Reihaneh Nasirigerdeh, Reza Kaissis, Georgios Rueckert, Daniel Dziugaite, Gintare Karolina Triantafillou, Eleni Machine Learning Machine unlearning refers to removing the influence of a specified subset of training data from a machine learning model, efficiently, after it has already been trained. This is important for key applications, including making the model more accurate by removing outdated, mislabeled, or poisoned data. In this work, we study localized unlearning, where the unlearning algorithm operates on a (small) identified subset of parameters. Drawing inspiration from the memorization literature, we propose an improved localization strategy that yields strong results when paired with existing unlearning algorithms. We also propose a new unlearning algorithm, Deletion by Example Localization (DEL), that resets the parameters deemed-to-be most critical according to our localization strategy, and then finetunes them. Our extensive experiments on different datasets, forget sets and metrics reveal that DEL sets a new state-of-the-art for unlearning metrics, against both localized and full-parameter methods, while modifying a small subset of parameters, and outperforms the state-of-the-art localized unlearning in terms of test accuracy too. |
| title | Improved Localized Machine Unlearning Through the Lens of Memorization |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2412.02432 |