Complexity of Adagrad and other first-order methods for nonconvex optimization problems with bounds constraints
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912099289006080 |
|---|---|
| author | Gratton, Serge Jerad, Sadok Toint, Philippe L. |
| author_facet | Gratton, Serge Jerad, Sadok Toint, Philippe L. |
| contents | A parametric class of trust-region algorithms for constrained nonconvex optimization is analyzed, where the objective function is never computed. By defining appropriate first-order stationarity criteria, we are able to extend the Adagrad method to the newly considered problem and retrieve the standard complexity rate of the projected gradient method that uses both the gradient and objective function values. Furthermore, we propose an additional iteration-dependent scaling with slightly inferior theoretical guarantees. In both cases, the bounds are essentially sharp, and curvature information can be used to compute the stepsize. Initial experimental results for noisy bound-constrained instances illustrate the benefits of the objective-free approach. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_15793 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Complexity of Adagrad and other first-order methods for nonconvex optimization problems with bounds constraints Gratton, Serge Jerad, Sadok Toint, Philippe L. Optimization and Control 90C60, 90C30, 90C15, 90C26, 49N30 F.2.1; G.1.6 A parametric class of trust-region algorithms for constrained nonconvex optimization is analyzed, where the objective function is never computed. By defining appropriate first-order stationarity criteria, we are able to extend the Adagrad method to the newly considered problem and retrieve the standard complexity rate of the projected gradient method that uses both the gradient and objective function values. Furthermore, we propose an additional iteration-dependent scaling with slightly inferior theoretical guarantees. In both cases, the bounds are essentially sharp, and curvature information can be used to compute the stepsize. Initial experimental results for noisy bound-constrained instances illustrate the benefits of the objective-free approach. |
| title | Complexity of Adagrad and other first-order methods for nonconvex optimization problems with bounds constraints |
| topic | Optimization and Control 90C60, 90C30, 90C15, 90C26, 49N30 F.2.1; G.1.6 |
| url | https://arxiv.org/abs/2406.15793 |