Block Majorization Minimization with Extrapolation and Application to $β$-NMF
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916928527794176 |
|---|---|
| author | Hien, Le Thi Khanh Leplat, Valentin Gillis, Nicolas |
| author_facet | Hien, Le Thi Khanh Leplat, Valentin Gillis, Nicolas |
| contents | We propose a Block Majorization Minimization method with Extrapolation (BMMe) for solving a class of multi-convex optimization problems. The extrapolation parameters of BMMe are updated using a novel adaptive update rule. By showing that block majorization minimization can be reformulated as a block mirror descent method, with the Bregman divergence adaptively updated at each iteration, we establish subsequential convergence for BMMe. We use this method to design efficient algorithms to tackle nonnegative matrix factorization problems with the $β$-divergences ($β$-NMF) for $β\in [1,2]$. These algorithms, which are multiplicative updates with extrapolation, benefit from our novel results that offer convergence guarantees. We also empirically illustrate the significant acceleration of BMMe for $β$-NMF through extensive experiments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_06646 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Block Majorization Minimization with Extrapolation and Application to $β$-NMF Hien, Le Thi Khanh Leplat, Valentin Gillis, Nicolas Machine Learning Numerical Analysis Signal Processing Optimization and Control We propose a Block Majorization Minimization method with Extrapolation (BMMe) for solving a class of multi-convex optimization problems. The extrapolation parameters of BMMe are updated using a novel adaptive update rule. By showing that block majorization minimization can be reformulated as a block mirror descent method, with the Bregman divergence adaptively updated at each iteration, we establish subsequential convergence for BMMe. We use this method to design efficient algorithms to tackle nonnegative matrix factorization problems with the $β$-divergences ($β$-NMF) for $β\in [1,2]$. These algorithms, which are multiplicative updates with extrapolation, benefit from our novel results that offer convergence guarantees. We also empirically illustrate the significant acceleration of BMMe for $β$-NMF through extensive experiments. |
| title | Block Majorization Minimization with Extrapolation and Application to $β$-NMF |
| topic | Machine Learning Numerical Analysis Signal Processing Optimization and Control |
| url | https://arxiv.org/abs/2401.06646 |