Improved Sample Complexity for Private Nonsmooth Nonconvex Optimization
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909641988898816 |
|---|---|
| author | Kornowski, Guy Liu, Daogao Talwar, Kunal |
| author_facet | Kornowski, Guy Liu, Daogao Talwar, Kunal |
| contents | We study differentially private (DP) optimization algorithms for stochastic and empirical objectives which are neither smooth nor convex, and propose methods that return a Goldstein-stationary point with sample complexity bounds that improve on existing works. We start by providing a single-pass $(ε,δ)$-DP algorithm that returns an $(α,β)$-stationary point as long as the dataset is of size $\widetildeΩ(\sqrt{d}/αβ^{3}+d/εαβ^{2})$, which is $Ω(\sqrt{d})$ times smaller than the algorithm of Zhang et al. [2024] for this task, where $d$ is the dimension. We then provide a multi-pass polynomial time algorithm which further improves the sample complexity to $\widetildeΩ\left(d/β^2+d^{3/4}/εα^{1/2}β^{3/2}\right)$, by designing a sample efficient ERM algorithm, and proving that Goldstein-stationary points generalize from the empirical loss to the population loss. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_05880 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Improved Sample Complexity for Private Nonsmooth Nonconvex Optimization Kornowski, Guy Liu, Daogao Talwar, Kunal Machine Learning Cryptography and Security Optimization and Control We study differentially private (DP) optimization algorithms for stochastic and empirical objectives which are neither smooth nor convex, and propose methods that return a Goldstein-stationary point with sample complexity bounds that improve on existing works. We start by providing a single-pass $(ε,δ)$-DP algorithm that returns an $(α,β)$-stationary point as long as the dataset is of size $\widetildeΩ(\sqrt{d}/αβ^{3}+d/εαβ^{2})$, which is $Ω(\sqrt{d})$ times smaller than the algorithm of Zhang et al. [2024] for this task, where $d$ is the dimension. We then provide a multi-pass polynomial time algorithm which further improves the sample complexity to $\widetildeΩ\left(d/β^2+d^{3/4}/εα^{1/2}β^{3/2}\right)$, by designing a sample efficient ERM algorithm, and proving that Goldstein-stationary points generalize from the empirical loss to the population loss. |
| title | Improved Sample Complexity for Private Nonsmooth Nonconvex Optimization |
| topic | Machine Learning Cryptography and Security Optimization and Control |
| url | https://arxiv.org/abs/2410.05880 |