Complexity and performance for two classes of noise-tolerant first-order algorithms
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2022
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910803666403328 |
|---|---|
| author | Gratton, S. Jerad, S. Toint, Ph. L. |
| author_facet | Gratton, S. Jerad, S. Toint, Ph. L. |
| contents | Two classes of algorithms for optimization in the presence of noise are presented, that do not require the evaluation of the objective function. The first generalizes the well-known Adagrad method. Its complexity is then analyzed as a function of its parameters. A second class of algorithms is then derived whose complexity is at least as good as that of the first class. Initial numerical experiments on finite-sum problems arising from deep-learning applications suggest that methods of the second class may outperform those of the first. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2203_01757 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Complexity and performance for two classes of noise-tolerant first-order algorithms Gratton, S. Jerad, S. Toint, Ph. L. Optimization and Control 90C60, 90C30, 90C15, 90C26, 49N30 F.2.1; G.1.6 Two classes of algorithms for optimization in the presence of noise are presented, that do not require the evaluation of the objective function. The first generalizes the well-known Adagrad method. Its complexity is then analyzed as a function of its parameters. A second class of algorithms is then derived whose complexity is at least as good as that of the first class. Initial numerical experiments on finite-sum problems arising from deep-learning applications suggest that methods of the second class may outperform those of the first. |
| title | Complexity and performance for two classes of noise-tolerant first-order algorithms |
| topic | Optimization and Control 90C60, 90C30, 90C15, 90C26, 49N30 F.2.1; G.1.6 |
| url | https://arxiv.org/abs/2203.01757 |