Generalized Metric Subregularity with Applications to High-Order Regularized Newton Methods
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909227777261568 |
|---|---|
| author | Li, Guoyin Mordukhovich, Boris Zhu, Jiangxing |
| author_facet | Li, Guoyin Mordukhovich, Boris Zhu, Jiangxing |
| contents | This paper pursues a twofold goal. First, we introduce and study in detail a new notion of variational analysis called generalized metric subregularity, which is a far-going extension of the conventional metric subregularity conditions. Our primary focus is on examining this concept concerning first-order and second-order stationary points. We develop an extended convergence framework that enables us to derive superlinear and quadratic convergence under the generalized metric subregularity condition, broadening the widely used KL convergence analysis framework. We present verifiable sufficient conditions to ensure the proposed generalized metric subregularity condition and provide examples demonstrating that the derived convergence rates are sharp. Second, we design a new high-order regularized Newton method with momentum steps, and apply the generalized metric subregularity to establish its superlinear convergence. Quadratic convergence is obtained under additional assumptions. Specifically, when applying the proposed method to solve the (nonconvex) over-parameterized compressed sensing model, we achieve global convergence with a quadratic local convergence rate towards a global minimizer under a strict complementarity condition. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_13207 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Generalized Metric Subregularity with Applications to High-Order Regularized Newton Methods Li, Guoyin Mordukhovich, Boris Zhu, Jiangxing Optimization and Control 49J53, 49M15, 90C26 This paper pursues a twofold goal. First, we introduce and study in detail a new notion of variational analysis called generalized metric subregularity, which is a far-going extension of the conventional metric subregularity conditions. Our primary focus is on examining this concept concerning first-order and second-order stationary points. We develop an extended convergence framework that enables us to derive superlinear and quadratic convergence under the generalized metric subregularity condition, broadening the widely used KL convergence analysis framework. We present verifiable sufficient conditions to ensure the proposed generalized metric subregularity condition and provide examples demonstrating that the derived convergence rates are sharp. Second, we design a new high-order regularized Newton method with momentum steps, and apply the generalized metric subregularity to establish its superlinear convergence. Quadratic convergence is obtained under additional assumptions. Specifically, when applying the proposed method to solve the (nonconvex) over-parameterized compressed sensing model, we achieve global convergence with a quadratic local convergence rate towards a global minimizer under a strict complementarity condition. |
| title | Generalized Metric Subregularity with Applications to High-Order Regularized Newton Methods |
| topic | Optimization and Control 49J53, 49M15, 90C26 |
| url | https://arxiv.org/abs/2406.13207 |