A Stochastic Block-coordinate Proximal Newton Method for Nonconvex Composite Minimization
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915887655682048 |
|---|---|
| author | Zhu, Hong Qian, Xun |
| author_facet | Zhu, Hong Qian, Xun |
| contents | This paper presents a stochastic block-coordinate proximal Newton method for minimizing the sum of a blockwise Lipschitz-continuously differentiable function and a separable nonsmooth convex function. At each iteration, the method randomly selects one block and approximately solves a strongly convex regularized quadratic subproblem built from a second-order local model of the smooth part of the objective function, with a backtracking line search to ensure monotonicity of the objective. Under mild sampling assumptions, we show that its convergence properties match those of the inexact proximal Newton method. We further develop a line-search-free variant, where the strongly convex regularized quadratic subproblem is constructed using the Lipschitz constant of the gradient of the smooth component. For this variant, under a suitable parameter setting, we establish the global convergence rate of the residual mapping as well as the superlinear convergence rate of the iterates under the metric \(q\)-subregularity property with \(q > 1\) of the residual mapping for nonconvex composite problems. Under a suitable parameter setting, a more restrictive condition on the Hessian approximation, and the Hölderian error bound condition (\(q\in(0, 1]\)) of the residual mapping, we also prove the local superlinear/quadratic convergence rate of both the residual mapping and the iterates for convex composite problems. Finally, numerical experiments are conducted to demonstrate the effectiveness and convergence behavior of the proposed algorithm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_18394 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Stochastic Block-coordinate Proximal Newton Method for Nonconvex Composite Minimization Zhu, Hong Qian, Xun Optimization and Control 90C26 49M15 65K05 This paper presents a stochastic block-coordinate proximal Newton method for minimizing the sum of a blockwise Lipschitz-continuously differentiable function and a separable nonsmooth convex function. At each iteration, the method randomly selects one block and approximately solves a strongly convex regularized quadratic subproblem built from a second-order local model of the smooth part of the objective function, with a backtracking line search to ensure monotonicity of the objective. Under mild sampling assumptions, we show that its convergence properties match those of the inexact proximal Newton method. We further develop a line-search-free variant, where the strongly convex regularized quadratic subproblem is constructed using the Lipschitz constant of the gradient of the smooth component. For this variant, under a suitable parameter setting, we establish the global convergence rate of the residual mapping as well as the superlinear convergence rate of the iterates under the metric \(q\)-subregularity property with \(q > 1\) of the residual mapping for nonconvex composite problems. Under a suitable parameter setting, a more restrictive condition on the Hessian approximation, and the Hölderian error bound condition (\(q\in(0, 1]\)) of the residual mapping, we also prove the local superlinear/quadratic convergence rate of both the residual mapping and the iterates for convex composite problems. Finally, numerical experiments are conducted to demonstrate the effectiveness and convergence behavior of the proposed algorithm. |
| title | A Stochastic Block-coordinate Proximal Newton Method for Nonconvex Composite Minimization |
| topic | Optimization and Control 90C26 49M15 65K05 |
| url | https://arxiv.org/abs/2412.18394 |