The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929743742369792 |
|---|---|
| author | Steinke, Thomas Nasr, Milad Ganesh, Arun Balle, Borja Choquette-Choo, Christopher A. Jagielski, Matthew Hayes, Jamie Thakurta, Abhradeep Guha Smith, Adam Terzis, Andreas |
| author_facet | Steinke, Thomas Nasr, Milad Ganesh, Arun Balle, Borja Choquette-Choo, Christopher A. Jagielski, Matthew Hayes, Jamie Thakurta, Abhradeep Guha Smith, Adam Terzis, Andreas |
| contents | We propose a simple heuristic privacy analysis of noisy clipped stochastic gradient descent (DP-SGD) in the setting where only the last iterate is released and the intermediate iterates remain hidden. Namely, our heuristic assumes a linear structure for the model.
We show experimentally that our heuristic is predictive of the outcome of privacy auditing applied to various training procedures. Thus it can be used prior to training as a rough estimate of the final privacy leakage. We also probe the limitations of our heuristic by providing some artificial counterexamples where it underestimates the privacy leakage.
The standard composition-based privacy analysis of DP-SGD effectively assumes that the adversary has access to all intermediate iterates, which is often unrealistic. However, this analysis remains the state of the art in practice. While our heuristic does not replace a rigorous privacy analysis, it illustrates the large gap between the best theoretical upper bounds and the privacy auditing lower bounds and sets a target for further work to improve the theoretical privacy analyses. We also empirically support our heuristic and show existing privacy auditing attacks are bounded by our heuristic analysis in both vision and language tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_06186 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD Steinke, Thomas Nasr, Milad Ganesh, Arun Balle, Borja Choquette-Choo, Christopher A. Jagielski, Matthew Hayes, Jamie Thakurta, Abhradeep Guha Smith, Adam Terzis, Andreas Cryptography and Security Machine Learning We propose a simple heuristic privacy analysis of noisy clipped stochastic gradient descent (DP-SGD) in the setting where only the last iterate is released and the intermediate iterates remain hidden. Namely, our heuristic assumes a linear structure for the model. We show experimentally that our heuristic is predictive of the outcome of privacy auditing applied to various training procedures. Thus it can be used prior to training as a rough estimate of the final privacy leakage. We also probe the limitations of our heuristic by providing some artificial counterexamples where it underestimates the privacy leakage. The standard composition-based privacy analysis of DP-SGD effectively assumes that the adversary has access to all intermediate iterates, which is often unrealistic. However, this analysis remains the state of the art in practice. While our heuristic does not replace a rigorous privacy analysis, it illustrates the large gap between the best theoretical upper bounds and the privacy auditing lower bounds and sets a target for further work to improve the theoretical privacy analyses. We also empirically support our heuristic and show existing privacy auditing attacks are bounded by our heuristic analysis in both vision and language tasks. |
| title | The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD |
| topic | Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2410.06186 |