Certification for Differentially Private Prediction in Gradient-Based Training
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913879604330496 |
|---|---|
| author | Wicker, Matthew Sosnin, Philip Shilov, Igor Janik, Adrianna Müller, Mark N. de Montjoye, Yves-Alexandre Weller, Adrian Tsay, Calvin |
| author_facet | Wicker, Matthew Sosnin, Philip Shilov, Igor Janik, Adrianna Müller, Mark N. de Montjoye, Yves-Alexandre Weller, Adrian Tsay, Calvin |
| contents | We study private prediction where differential privacy is achieved by adding noise to the outputs of a non-private model. Existing methods rely on noise proportional to the global sensitivity of the model, often resulting in sub-optimal privacy-utility trade-offs compared to private training. We introduce a novel approach for computing dataset-specific upper bounds on prediction sensitivity by leveraging convex relaxation and bound propagation techniques. By combining these bounds with the smooth sensitivity mechanism, we significantly improve the privacy analysis of private prediction compared to global sensitivity-based approaches. Experimental results across real-world datasets in medical image classification and natural language processing demonstrate that our sensitivity bounds are can be orders of magnitude tighter than global sensitivity. Our approach provides a strong basis for the development of novel privacy preserving technologies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_13433 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Certification for Differentially Private Prediction in Gradient-Based Training Wicker, Matthew Sosnin, Philip Shilov, Igor Janik, Adrianna Müller, Mark N. de Montjoye, Yves-Alexandre Weller, Adrian Tsay, Calvin Machine Learning Artificial Intelligence We study private prediction where differential privacy is achieved by adding noise to the outputs of a non-private model. Existing methods rely on noise proportional to the global sensitivity of the model, often resulting in sub-optimal privacy-utility trade-offs compared to private training. We introduce a novel approach for computing dataset-specific upper bounds on prediction sensitivity by leveraging convex relaxation and bound propagation techniques. By combining these bounds with the smooth sensitivity mechanism, we significantly improve the privacy analysis of private prediction compared to global sensitivity-based approaches. Experimental results across real-world datasets in medical image classification and natural language processing demonstrate that our sensitivity bounds are can be orders of magnitude tighter than global sensitivity. Our approach provides a strong basis for the development of novel privacy preserving technologies. |
| title | Certification for Differentially Private Prediction in Gradient-Based Training |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2406.13433 |