Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.02079 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914004735098880 |
|---|---|
| author | Hamad, Fadi Hinder, Oliver |
| author_facet | Hamad, Fadi Hinder, Oliver |
| contents | We present an adaptive trust-region method for unconstrained optimization that allows inexact solutions to the trust-region subproblems. Our method is a simple variant of the classical trust-region method of \citet{sorensen1982newton}. The method achieves the best possible convergence bound up to an additive log factor, for finding an $ε$-approximate stationary point, i.e., $O( Δ_f L^{1/2} ε^{-3/2}) + \tilde{O}(1)$ iterations where $L$ is the Lipschitz constant of the Hessian, $Δ_f$ is the optimality gap, and $ε$ is the termination tolerance for the gradient norm. This improves over existing trust-region methods whose worst-case bound is at least a factor of $L$ worse. We compare our performance with state-of-the-art trust-region (TRU) and cubic regularization (ARC) methods from the GALAHAD library on the CUTEst benchmark set on problems with more than 100 variables. We use fewer function, gradient, and Hessian evaluations than these methods. For instance, our algorithm's median number of gradient evaluations is $23$ compared to $36$ for TRU and $29$ for ARC.
Compared to the conference version of this paper \cite{hamad2022consistently}, our revised method includes several practical enhancements. These modifications dramatically improved performance, including an order of magnitude reduction in the shifted geometric mean of wall-clock times. We also show it suffices for the second derivatives to be locally Lipschitz to guarantee that either the minimum gradient norm converges to zero or the objective value tends towards negative infinity, even when the iterates diverge. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_02079 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A simple and practical adaptive trust-region method Hamad, Fadi Hinder, Oliver Optimization and Control We present an adaptive trust-region method for unconstrained optimization that allows inexact solutions to the trust-region subproblems. Our method is a simple variant of the classical trust-region method of \citet{sorensen1982newton}. The method achieves the best possible convergence bound up to an additive log factor, for finding an $ε$-approximate stationary point, i.e., $O( Δ_f L^{1/2} ε^{-3/2}) + \tilde{O}(1)$ iterations where $L$ is the Lipschitz constant of the Hessian, $Δ_f$ is the optimality gap, and $ε$ is the termination tolerance for the gradient norm. This improves over existing trust-region methods whose worst-case bound is at least a factor of $L$ worse. We compare our performance with state-of-the-art trust-region (TRU) and cubic regularization (ARC) methods from the GALAHAD library on the CUTEst benchmark set on problems with more than 100 variables. We use fewer function, gradient, and Hessian evaluations than these methods. For instance, our algorithm's median number of gradient evaluations is $23$ compared to $36$ for TRU and $29$ for ARC. Compared to the conference version of this paper \cite{hamad2022consistently}, our revised method includes several practical enhancements. These modifications dramatically improved performance, including an order of magnitude reduction in the shifted geometric mean of wall-clock times. We also show it suffices for the second derivatives to be locally Lipschitz to guarantee that either the minimum gradient norm converges to zero or the objective value tends towards negative infinity, even when the iterates diverge. |
| title | A simple and practical adaptive trust-region method |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2412.02079 |