Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2402.02359 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911770832011264 |
|---|---|
| author | Liu, Zhuanghua Luo, Luo Low, Bryan Kian Hsiang |
| author_facet | Liu, Zhuanghua Luo, Luo Low, Bryan Kian Hsiang |
| contents | We consider the finite-sum optimization problem, where each component function is strongly convex and has Lipschitz continuous gradient and Hessian. The recently proposed incremental quasi-Newton method is based on BFGS update and achieves a local superlinear convergence rate that is dependent on the condition number of the problem. This paper proposes a more efficient quasi-Newton method by incorporating the symmetric rank-1 update into the incremental framework, which results in the condition-number-free local superlinear convergence rate. Furthermore, we can boost our method by applying the block update on the Hessian approximation, which leads to an even faster local convergence rate. The numerical experiments show the proposed methods significantly outperform the baseline methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_02359 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Incremental Quasi-Newton Methods with Faster Superlinear Convergence Rates Liu, Zhuanghua Luo, Luo Low, Bryan Kian Hsiang Optimization and Control Machine Learning We consider the finite-sum optimization problem, where each component function is strongly convex and has Lipschitz continuous gradient and Hessian. The recently proposed incremental quasi-Newton method is based on BFGS update and achieves a local superlinear convergence rate that is dependent on the condition number of the problem. This paper proposes a more efficient quasi-Newton method by incorporating the symmetric rank-1 update into the incremental framework, which results in the condition-number-free local superlinear convergence rate. Furthermore, we can boost our method by applying the block update on the Hessian approximation, which leads to an even faster local convergence rate. The numerical experiments show the proposed methods significantly outperform the baseline methods. |
| title | Incremental Quasi-Newton Methods with Faster Superlinear Convergence Rates |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2402.02359 |