Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.04006 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910690862694400 |
|---|---|
| author | Chen, Shi Li, Qin Tse, Oliver Wright, Stephen J. |
| author_facet | Chen, Shi Li, Qin Tse, Oliver Wright, Stephen J. |
| contents | The acceleration of gradient-based optimization methods is a subject of significant practical and theoretical importance, particularly within machine learning applications. While much attention has been directed towards optimizing within Euclidean space, the need to optimize over spaces of probability measures in machine learning motivates exploration of accelerated gradient methods in this context too. To this end, we introduce a Hamiltonian-flow approach analogous to momentum-based approaches in Euclidean space. We demonstrate that, in the continuous-time setting, algorithms based on this approach can achieve convergence rates of arbitrarily high order. We complement our findings with numerical examples. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_04006 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Accelerating optimization over the space of probability measures Chen, Shi Li, Qin Tse, Oliver Wright, Stephen J. Optimization and Control Machine Learning The acceleration of gradient-based optimization methods is a subject of significant practical and theoretical importance, particularly within machine learning applications. While much attention has been directed towards optimizing within Euclidean space, the need to optimize over spaces of probability measures in machine learning motivates exploration of accelerated gradient methods in this context too. To this end, we introduce a Hamiltonian-flow approach analogous to momentum-based approaches in Euclidean space. We demonstrate that, in the continuous-time setting, algorithms based on this approach can achieve convergence rates of arbitrarily high order. We complement our findings with numerical examples. |
| title | Accelerating optimization over the space of probability measures |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2310.04006 |