Optimal Algorithms for Stochastic Complementary Composite Minimization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2022
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910305106264064 |
|---|---|
| author | d'Aspremont, Alexandre Guzmán, Cristóbal Lezane, Clément |
| author_facet | d'Aspremont, Alexandre Guzmán, Cristóbal Lezane, Clément |
| contents | Inspired by regularization techniques in statistics and machine learning, we study complementary composite minimization in the stochastic setting. This problem corresponds to the minimization of the sum of a (weakly) smooth function endowed with a stochastic first-order oracle, and a structured uniformly convex (possibly nonsmooth and non-Lipschitz) regularization term. Despite intensive work on closely related settings, prior to our work no complexity bounds for this problem were known. We close this gap by providing novel excess risk bounds, both in expectation and with high probability. Our algorithms are nearly optimal, which we prove via novel lower complexity bounds for this class of problems. We conclude by providing numerical results comparing our methods to the state of the art. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2211_01758 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Optimal Algorithms for Stochastic Complementary Composite Minimization d'Aspremont, Alexandre Guzmán, Cristóbal Lezane, Clément Machine Learning Optimization and Control Inspired by regularization techniques in statistics and machine learning, we study complementary composite minimization in the stochastic setting. This problem corresponds to the minimization of the sum of a (weakly) smooth function endowed with a stochastic first-order oracle, and a structured uniformly convex (possibly nonsmooth and non-Lipschitz) regularization term. Despite intensive work on closely related settings, prior to our work no complexity bounds for this problem were known. We close this gap by providing novel excess risk bounds, both in expectation and with high probability. Our algorithms are nearly optimal, which we prove via novel lower complexity bounds for this class of problems. We conclude by providing numerical results comparing our methods to the state of the art. |
| title | Optimal Algorithms for Stochastic Complementary Composite Minimization |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2211.01758 |