Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2408.05723 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917746819727360 |
|---|---|
| author | Tao, Wenqi Ling, Huaming Shi, Zuoqiang Wang, Bao |
| author_facet | Tao, Wenqi Ling, Huaming Shi, Zuoqiang Wang, Bao |
| contents | Protecting data privacy in deep learning (DL) is of crucial importance. Several celebrated privacy notions have been established and used for privacy-preserving DL. However, many existing mechanisms achieve privacy at the cost of significant utility degradation and computational overhead. In this paper, we propose a stochastic differential equation-based residual perturbation for privacy-preserving DL, which injects Gaussian noise into each residual mapping of ResNets. Theoretically, we prove that residual perturbation guarantees differential privacy (DP) and reduces the generalization gap of DL. Empirically, we show that residual perturbation is computationally efficient and outperforms the state-of-the-art differentially private stochastic gradient descent (DPSGD) in utility maintenance without sacrificing membership privacy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_05723 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Deep Learning with Data Privacy via Residual Perturbation Tao, Wenqi Ling, Huaming Shi, Zuoqiang Wang, Bao Machine Learning Cryptography and Security Computer Vision and Pattern Recognition Protecting data privacy in deep learning (DL) is of crucial importance. Several celebrated privacy notions have been established and used for privacy-preserving DL. However, many existing mechanisms achieve privacy at the cost of significant utility degradation and computational overhead. In this paper, we propose a stochastic differential equation-based residual perturbation for privacy-preserving DL, which injects Gaussian noise into each residual mapping of ResNets. Theoretically, we prove that residual perturbation guarantees differential privacy (DP) and reduces the generalization gap of DL. Empirically, we show that residual perturbation is computationally efficient and outperforms the state-of-the-art differentially private stochastic gradient descent (DPSGD) in utility maintenance without sacrificing membership privacy. |
| title | Deep Learning with Data Privacy via Residual Perturbation |
| topic | Machine Learning Cryptography and Security Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.05723 |