Label Smoothing Improves Machine Unlearning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929382614892544 |
|---|---|
| author | Di, Zonglin Zhu, Zhaowei Jia, Jinghan Liu, Jiancheng Takhirov, Zafar Jiang, Bo Yao, Yuanshun Liu, Sijia Liu, Yang |
| author_facet | Di, Zonglin Zhu, Zhaowei Jia, Jinghan Liu, Jiancheng Takhirov, Zafar Jiang, Bo Yao, Yuanshun Liu, Sijia Liu, Yang |
| contents | The objective of machine unlearning (MU) is to eliminate previously learned data from a model. However, it is challenging to strike a balance between computation cost and performance when using existing MU techniques. Taking inspiration from the influence of label smoothing on model confidence and differential privacy, we propose a simple gradient-based MU approach that uses an inverse process of label smoothing. This work introduces UGradSL, a simple, plug-and-play MU approach that uses smoothed labels. We provide theoretical analyses demonstrating why properly introducing label smoothing improves MU performance. We conducted extensive experiments on six datasets of various sizes and different modalities, demonstrating the effectiveness and robustness of our proposed method. The consistent improvement in MU performance is only at a marginal cost of additional computations. For instance, UGradSL improves over the gradient ascent MU baseline by 66% unlearning accuracy without sacrificing unlearning efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_07698 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Label Smoothing Improves Machine Unlearning Di, Zonglin Zhu, Zhaowei Jia, Jinghan Liu, Jiancheng Takhirov, Zafar Jiang, Bo Yao, Yuanshun Liu, Sijia Liu, Yang Machine Learning The objective of machine unlearning (MU) is to eliminate previously learned data from a model. However, it is challenging to strike a balance between computation cost and performance when using existing MU techniques. Taking inspiration from the influence of label smoothing on model confidence and differential privacy, we propose a simple gradient-based MU approach that uses an inverse process of label smoothing. This work introduces UGradSL, a simple, plug-and-play MU approach that uses smoothed labels. We provide theoretical analyses demonstrating why properly introducing label smoothing improves MU performance. We conducted extensive experiments on six datasets of various sizes and different modalities, demonstrating the effectiveness and robustness of our proposed method. The consistent improvement in MU performance is only at a marginal cost of additional computations. For instance, UGradSL improves over the gradient ascent MU baseline by 66% unlearning accuracy without sacrificing unlearning efficiency. |
| title | Label Smoothing Improves Machine Unlearning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2406.07698 |