Black-Box Uniform Stability for Non-Euclidean Empirical Risk Minimization
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915073745747968 |
|---|---|
| author | Vary, Simon Martínez-Rubio, David Rebeschini, Patrick |
| author_facet | Vary, Simon Martínez-Rubio, David Rebeschini, Patrick |
| contents | We study first-order algorithms that are uniformly stable for empirical risk minimization (ERM) problems that are convex and smooth with respect to $p$-norms, $p \geq 1$. We propose a black-box reduction method that, by employing properties of uniformly convex regularizers, turns an optimization algorithm for Hölder smooth convex losses into a uniformly stable learning algorithm with optimal statistical risk bounds on the excess risk, up to a constant factor depending on $p$. Achieving a black-box reduction for uniform stability was posed as an open question by (Attia and Koren, 2022), which had solved the Euclidean case $p=2$. We explore applications that leverage non-Euclidean geometry in addressing binary classification problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_15956 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Black-Box Uniform Stability for Non-Euclidean Empirical Risk Minimization Vary, Simon Martínez-Rubio, David Rebeschini, Patrick Machine Learning Optimization and Control 68Q32, 90C25 G.1.6; I.2.6 We study first-order algorithms that are uniformly stable for empirical risk minimization (ERM) problems that are convex and smooth with respect to $p$-norms, $p \geq 1$. We propose a black-box reduction method that, by employing properties of uniformly convex regularizers, turns an optimization algorithm for Hölder smooth convex losses into a uniformly stable learning algorithm with optimal statistical risk bounds on the excess risk, up to a constant factor depending on $p$. Achieving a black-box reduction for uniform stability was posed as an open question by (Attia and Koren, 2022), which had solved the Euclidean case $p=2$. We explore applications that leverage non-Euclidean geometry in addressing binary classification problems. |
| title | Black-Box Uniform Stability for Non-Euclidean Empirical Risk Minimization |
| topic | Machine Learning Optimization and Control 68Q32, 90C25 G.1.6; I.2.6 |
| url | https://arxiv.org/abs/2412.15956 |