Leveraging Per-Instance Privacy for Machine Unlearning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909622190735360 |
|---|---|
| author | Sepahvand, Nazanin Mohammadi Thudi, Anvith Isik, Berivan Bhattacharyya, Ashmita Papernot, Nicolas Triantafillou, Eleni Roy, Daniel M. Dziugaite, Gintare Karolina |
| author_facet | Sepahvand, Nazanin Mohammadi Thudi, Anvith Isik, Berivan Bhattacharyya, Ashmita Papernot, Nicolas Triantafillou, Eleni Roy, Daniel M. Dziugaite, Gintare Karolina |
| contents | We present a principled, per-instance approach to quantifying the difficulty of unlearning via fine-tuning. We begin by sharpening an analysis of noisy gradient descent for unlearning (Chien et al., 2024), obtaining a better utility-unlearning tradeoff by replacing worst-case privacy loss bounds with per-instance privacy losses (Thudi et al., 2024), each of which bounds the (Renyi) divergence to retraining without an individual data point. To demonstrate the practical applicability of our theory, we present empirical results showing that our theoretical predictions are born out both for Stochastic Gradient Langevin Dynamics (SGLD) as well as for standard fine-tuning without explicit noise. We further demonstrate that per-instance privacy losses correlate well with several existing data difficulty metrics, while also identifying harder groups of data points, and introduce novel evaluation methods based on loss barriers. All together, our findings provide a foundation for more efficient and adaptive unlearning strategies tailored to the unique properties of individual data points. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_18786 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Leveraging Per-Instance Privacy for Machine Unlearning Sepahvand, Nazanin Mohammadi Thudi, Anvith Isik, Berivan Bhattacharyya, Ashmita Papernot, Nicolas Triantafillou, Eleni Roy, Daniel M. Dziugaite, Gintare Karolina Machine Learning We present a principled, per-instance approach to quantifying the difficulty of unlearning via fine-tuning. We begin by sharpening an analysis of noisy gradient descent for unlearning (Chien et al., 2024), obtaining a better utility-unlearning tradeoff by replacing worst-case privacy loss bounds with per-instance privacy losses (Thudi et al., 2024), each of which bounds the (Renyi) divergence to retraining without an individual data point. To demonstrate the practical applicability of our theory, we present empirical results showing that our theoretical predictions are born out both for Stochastic Gradient Langevin Dynamics (SGLD) as well as for standard fine-tuning without explicit noise. We further demonstrate that per-instance privacy losses correlate well with several existing data difficulty metrics, while also identifying harder groups of data points, and introduce novel evaluation methods based on loss barriers. All together, our findings provide a foundation for more efficient and adaptive unlearning strategies tailored to the unique properties of individual data points. |
| title | Leveraging Per-Instance Privacy for Machine Unlearning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.18786 |