Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913543590248448 |
|---|---|
| author | Huang, Yangsibo Liu, Daogao Chua, Lynn Ghazi, Badih Kamath, Pritish Kumar, Ravi Manurangsi, Pasin Nasr, Milad Sinha, Amer Zhang, Chiyuan |
| author_facet | Huang, Yangsibo Liu, Daogao Chua, Lynn Ghazi, Badih Kamath, Pritish Kumar, Ravi Manurangsi, Pasin Nasr, Milad Sinha, Amer Zhang, Chiyuan |
| contents | Machine unlearning algorithms, designed for selective removal of training data from models, have emerged as a promising approach to growing privacy concerns. In this work, we expose a critical yet underexplored vulnerability in the deployment of unlearning systems: the assumption that the data requested for removal is always part of the original training set. We present a threat model where an attacker can degrade model accuracy by submitting adversarial unlearning requests for data not present in the training set. We propose white-box and black-box attack algorithms and evaluate them through a case study on image classification tasks using the CIFAR-10 and ImageNet datasets, targeting a family of widely used unlearning methods. Our results show extremely poor test accuracy following the attack: 3.6% on CIFAR-10 and 0.4% on ImageNet for white-box attacks, and 8.5% on CIFAR-10 and 1.3% on ImageNet for black-box attacks. Additionally, we evaluate various verification mechanisms to detect the legitimacy of unlearning requests and reveal the challenges in verification, as most of the mechanisms fail to detect stealthy attacks without severely impairing their ability to process valid requests. These findings underscore the urgent need for research on more robust request verification methods and unlearning protocols, should the deployment of machine unlearning systems become more prevalent in the future. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_09591 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy Huang, Yangsibo Liu, Daogao Chua, Lynn Ghazi, Badih Kamath, Pritish Kumar, Ravi Manurangsi, Pasin Nasr, Milad Sinha, Amer Zhang, Chiyuan Cryptography and Security Machine unlearning algorithms, designed for selective removal of training data from models, have emerged as a promising approach to growing privacy concerns. In this work, we expose a critical yet underexplored vulnerability in the deployment of unlearning systems: the assumption that the data requested for removal is always part of the original training set. We present a threat model where an attacker can degrade model accuracy by submitting adversarial unlearning requests for data not present in the training set. We propose white-box and black-box attack algorithms and evaluate them through a case study on image classification tasks using the CIFAR-10 and ImageNet datasets, targeting a family of widely used unlearning methods. Our results show extremely poor test accuracy following the attack: 3.6% on CIFAR-10 and 0.4% on ImageNet for white-box attacks, and 8.5% on CIFAR-10 and 1.3% on ImageNet for black-box attacks. Additionally, we evaluate various verification mechanisms to detect the legitimacy of unlearning requests and reveal the challenges in verification, as most of the mechanisms fail to detect stealthy attacks without severely impairing their ability to process valid requests. These findings underscore the urgent need for research on more robust request verification methods and unlearning protocols, should the deployment of machine unlearning systems become more prevalent in the future. |
| title | Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy |
| topic | Cryptography and Security |
| url | https://arxiv.org/abs/2410.09591 |