First-order majorization-minimization meets high-order majorant: Boosted inexact high-order forward-backward method
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909870012235776 |
|---|---|
| author | Kabgani, Alireza Ahookhosh, Masoud |
| author_facet | Kabgani, Alireza Ahookhosh, Masoud |
| contents | This paper introduces a first-order majorization-minimization framework based on a high-order majorant for continuous functions, incorporating a non-quadratic regularization term of degree $p>1$. Notably, it is shown to be valid if and only if the function is $p$-paraconcave, thus extending beyond Lipschitz and Hölder gradient continuity for $p \in (1,2]$, and implying concavity for $p>2$. In the smooth setting, this majorant recovers a variant of the classical descent lemma with quadratic regularization. Building on this foundation, we develop a high-order inexact forward-backward algorithm (HiFBA) and its line-search-accelerated variant, named Boosted HiFBA. For convergence analysis, we introduce a high-order forward-backward envelope (HiFBE), which serves as a Lyapunov function. We establish subsequential convergence under suitable inexactness conditions, and we prove global convergence with linear rates for functions satisfying the Kurdyka-Łojasiewicz inequality. Our preliminary experiments on linear inverse problems and regularized nonnegative matrix factorization highlight the efficiency of HiFBA and its boosted variant, demonstrating their potential for solving challenging nonconvex optimization problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_22231 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | First-order majorization-minimization meets high-order majorant: Boosted inexact high-order forward-backward method Kabgani, Alireza Ahookhosh, Masoud Optimization and Control 90C26, 65K05, 49J52, 90C30, 49M27 This paper introduces a first-order majorization-minimization framework based on a high-order majorant for continuous functions, incorporating a non-quadratic regularization term of degree $p>1$. Notably, it is shown to be valid if and only if the function is $p$-paraconcave, thus extending beyond Lipschitz and Hölder gradient continuity for $p \in (1,2]$, and implying concavity for $p>2$. In the smooth setting, this majorant recovers a variant of the classical descent lemma with quadratic regularization. Building on this foundation, we develop a high-order inexact forward-backward algorithm (HiFBA) and its line-search-accelerated variant, named Boosted HiFBA. For convergence analysis, we introduce a high-order forward-backward envelope (HiFBE), which serves as a Lyapunov function. We establish subsequential convergence under suitable inexactness conditions, and we prove global convergence with linear rates for functions satisfying the Kurdyka-Łojasiewicz inequality. Our preliminary experiments on linear inverse problems and regularized nonnegative matrix factorization highlight the efficiency of HiFBA and its boosted variant, demonstrating their potential for solving challenging nonconvex optimization problems. |
| title | First-order majorization-minimization meets high-order majorant: Boosted inexact high-order forward-backward method |
| topic | Optimization and Control 90C26, 65K05, 49J52, 90C30, 49M27 |
| url | https://arxiv.org/abs/2510.22231 |