Recursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimization
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918093240926208 |
|---|---|
| author | Gratton, Serge Kopaničáková, Alena Toint, Philippe |
| author_facet | Gratton, Serge Kopaničáková, Alena Toint, Philippe |
| contents | Two OFFO (Objective-Function Free Optimization) noise tolerant algorithms are presented that handle bound constraints, inexact gradients and use second-order information when available.The first is a multi-level method exploiting a hierarchical description of the problem and the second is a domain-decomposition method covering the standard addditive Schwarz decompositions. Both are generalizations of the first-order AdaGrad algorithm for unconstrained optimization. Because these algorithms share a common theoretical framework, a single convergence/complexity theory is provided which covers them both. Its main result is that, with high probability, both methods need at most $O(ε^{-2})$ iterations and noisy gradient evaluations to compute an $ε$-approximate first-order critical point of the bound-constrained problem. Extensive numerical experiments are discussed on applications ranging from PDE-based problems to deep neural network training, illustrating their remarkable computational efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_11513 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Recursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimization Gratton, Serge Kopaničáková, Alena Toint, Philippe Optimization and Control Artificial Intelligence Numerical Analysis 49K20, 65M55, 65Y20, 68Q25, 68T05, 90C26, 90C30 F.2.1; G.1.8; I.2.5 Two OFFO (Objective-Function Free Optimization) noise tolerant algorithms are presented that handle bound constraints, inexact gradients and use second-order information when available.The first is a multi-level method exploiting a hierarchical description of the problem and the second is a domain-decomposition method covering the standard addditive Schwarz decompositions. Both are generalizations of the first-order AdaGrad algorithm for unconstrained optimization. Because these algorithms share a common theoretical framework, a single convergence/complexity theory is provided which covers them both. Its main result is that, with high probability, both methods need at most $O(ε^{-2})$ iterations and noisy gradient evaluations to compute an $ε$-approximate first-order critical point of the bound-constrained problem. Extensive numerical experiments are discussed on applications ranging from PDE-based problems to deep neural network training, illustrating their remarkable computational efficiency. |
| title | Recursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimization |
| topic | Optimization and Control Artificial Intelligence Numerical Analysis 49K20, 65M55, 65Y20, 68Q25, 68T05, 90C26, 90C30 F.2.1; G.1.8; I.2.5 |
| url | https://arxiv.org/abs/2507.11513 |