Majorization-Minimization-Based Levenberg--Marquardt Method for Constrained Nonlinear Least Squares
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2020
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909202173132800 |
|---|---|
| author | Marumo, Naoki Okuno, Takayuki Takeda, Akiko |
| author_facet | Marumo, Naoki Okuno, Takayuki Takeda, Akiko |
| contents | A new Levenberg--Marquardt (LM) method for solving nonlinear least squares problems with convex constraints is described. Various versions of the LM method have been proposed, their main differences being in the choice of a damping parameter. In this paper, we propose a new rule for updating the parameter so as to achieve both global and local convergence even under the presence of a convex constraint set. The key to our results is a new perspective of the LM method from majorization-minimization methods. Specifically, we show that if the damping parameter is set in a specific way, the objective function of the standard subproblem in LM methods becomes an upper bound on the original objective function under certain standard assumptions.
Our method solves a sequence of the subproblems approximately using an (accelerated) projected gradient method. It finds an $ε$-stationary point after $O(ε^{-2})$ computation and achieves local quadratic convergence for zero-residual problems under a local error bound condition. Numerical results on compressed sensing and matrix factorization show that our method converges faster in many cases than existing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2004_08259 |
| institution | arXiv |
| publishDate | 2020 |
| record_format | arxiv |
| spellingShingle | Majorization-Minimization-Based Levenberg--Marquardt Method for Constrained Nonlinear Least Squares Marumo, Naoki Okuno, Takayuki Takeda, Akiko Optimization and Control 65K05, 90C30 G.1.6; G.1.5 A new Levenberg--Marquardt (LM) method for solving nonlinear least squares problems with convex constraints is described. Various versions of the LM method have been proposed, their main differences being in the choice of a damping parameter. In this paper, we propose a new rule for updating the parameter so as to achieve both global and local convergence even under the presence of a convex constraint set. The key to our results is a new perspective of the LM method from majorization-minimization methods. Specifically, we show that if the damping parameter is set in a specific way, the objective function of the standard subproblem in LM methods becomes an upper bound on the original objective function under certain standard assumptions. Our method solves a sequence of the subproblems approximately using an (accelerated) projected gradient method. It finds an $ε$-stationary point after $O(ε^{-2})$ computation and achieves local quadratic convergence for zero-residual problems under a local error bound condition. Numerical results on compressed sensing and matrix factorization show that our method converges faster in many cases than existing methods. |
| title | Majorization-Minimization-Based Levenberg--Marquardt Method for Constrained Nonlinear Least Squares |
| topic | Optimization and Control 65K05, 90C30 G.1.6; G.1.5 |
| url | https://arxiv.org/abs/2004.08259 |