Differentially Private Gradient Flow based on the Sliced Wasserstein Distance
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917895802454016 |
|---|---|
| author | Sebag, Ilana Pydi, Muni Sreenivas Franceschi, Jean-Yves Rakotomamonjy, Alain Gartrell, Mike Atif, Jamal Allauzen, Alexandre |
| author_facet | Sebag, Ilana Pydi, Muni Sreenivas Franceschi, Jean-Yves Rakotomamonjy, Alain Gartrell, Mike Atif, Jamal Allauzen, Alexandre |
| contents | Safeguarding privacy in sensitive training data is paramount, particularly in the context of generative modeling. This can be achieved through either differentially private stochastic gradient descent or a differentially private metric for training models or generators. In this paper, we introduce a novel differentially private generative modeling approach based on a gradient flow in the space of probability measures. To this end, we define the gradient flow of the Gaussian-smoothed Sliced Wasserstein Distance, including the associated stochastic differential equation (SDE). By discretizing and defining a numerical scheme for solving this SDE, we demonstrate the link between smoothing and differential privacy based on a Gaussian mechanism, due to a specific form of the SDE's drift term. We then analyze the differential privacy guarantee of our gradient flow, which accounts for both the smoothing and the Wiener process introduced by the SDE itself. Experiments show that our proposed model can generate higher-fidelity data at a low privacy budget compared to a generator-based model, offering a promising alternative. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_08227 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Differentially Private Gradient Flow based on the Sliced Wasserstein Distance Sebag, Ilana Pydi, Muni Sreenivas Franceschi, Jean-Yves Rakotomamonjy, Alain Gartrell, Mike Atif, Jamal Allauzen, Alexandre Machine Learning Cryptography and Security Safeguarding privacy in sensitive training data is paramount, particularly in the context of generative modeling. This can be achieved through either differentially private stochastic gradient descent or a differentially private metric for training models or generators. In this paper, we introduce a novel differentially private generative modeling approach based on a gradient flow in the space of probability measures. To this end, we define the gradient flow of the Gaussian-smoothed Sliced Wasserstein Distance, including the associated stochastic differential equation (SDE). By discretizing and defining a numerical scheme for solving this SDE, we demonstrate the link between smoothing and differential privacy based on a Gaussian mechanism, due to a specific form of the SDE's drift term. We then analyze the differential privacy guarantee of our gradient flow, which accounts for both the smoothing and the Wiener process introduced by the SDE itself. Experiments show that our proposed model can generate higher-fidelity data at a low privacy budget compared to a generator-based model, offering a promising alternative. |
| title | Differentially Private Gradient Flow based on the Sliced Wasserstein Distance |
| topic | Machine Learning Cryptography and Security |
| url | https://arxiv.org/abs/2312.08227 |