Unlearning via Sparse Representations
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913541154406400 |
|---|---|
| author | Shah, Vedant Träuble, Frederik Malik, Ashish Larochelle, Hugo Mozer, Michael Arora, Sanjeev Bengio, Yoshua Goyal, Anirudh |
| author_facet | Shah, Vedant Träuble, Frederik Malik, Ashish Larochelle, Hugo Mozer, Michael Arora, Sanjeev Bengio, Yoshua Goyal, Anirudh |
| contents | Machine \emph{unlearning}, which involves erasing knowledge about a \emph{forget set} from a trained model, can prove to be costly and infeasible by existing techniques. We propose a nearly compute-free zero-shot unlearning technique based on a discrete representational bottleneck. We show that the proposed technique efficiently unlearns the forget set and incurs negligible damage to the model's performance on the rest of the data set. We evaluate the proposed technique on the problem of \textit{class unlearning} using three datasets: CIFAR-10, CIFAR-100, and LACUNA-100. We compare the proposed technique to SCRUB, a state-of-the-art approach which uses knowledge distillation for unlearning. Across all three datasets, the proposed technique performs as well as, if not better than SCRUB while incurring almost no computational cost. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_15268 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Unlearning via Sparse Representations Shah, Vedant Träuble, Frederik Malik, Ashish Larochelle, Hugo Mozer, Michael Arora, Sanjeev Bengio, Yoshua Goyal, Anirudh Machine Learning Artificial Intelligence Machine \emph{unlearning}, which involves erasing knowledge about a \emph{forget set} from a trained model, can prove to be costly and infeasible by existing techniques. We propose a nearly compute-free zero-shot unlearning technique based on a discrete representational bottleneck. We show that the proposed technique efficiently unlearns the forget set and incurs negligible damage to the model's performance on the rest of the data set. We evaluate the proposed technique on the problem of \textit{class unlearning} using three datasets: CIFAR-10, CIFAR-100, and LACUNA-100. We compare the proposed technique to SCRUB, a state-of-the-art approach which uses knowledge distillation for unlearning. Across all three datasets, the proposed technique performs as well as, if not better than SCRUB while incurring almost no computational cost. |
| title | Unlearning via Sparse Representations |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2311.15268 |