prunAdag: an adaptive pruning-aware gradient method
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929711915991040 |
|---|---|
| author | Porcelli, Margherita Seraghiti, Giovanni Toint, Philippe L. |
| author_facet | Porcelli, Margherita Seraghiti, Giovanni Toint, Philippe L. |
| contents | A pruning-aware adaptive gradient method is proposed which classifies the variables in two sets before updating them using different strategies. This technique extends the ``relevant/irrelevant" approach of Ding (2019) and Zimmer et al. (2022) and allows a posteriori sparsification of the solution of model parameter fitting problems. The new method is proved to be convergent with a global rate of decrease of the averaged gradient's norm of the form $\calO(\log(k)/\sqrt{k+1})$. Numerical experiments on several applications show that it is competitive. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_08308 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | prunAdag: an adaptive pruning-aware gradient method Porcelli, Margherita Seraghiti, Giovanni Toint, Philippe L. Optimization and Control 49M05, 49M15, 65K10, 68Q25, 90C26 F.2.1; I.2.6; G.1.6 A pruning-aware adaptive gradient method is proposed which classifies the variables in two sets before updating them using different strategies. This technique extends the ``relevant/irrelevant" approach of Ding (2019) and Zimmer et al. (2022) and allows a posteriori sparsification of the solution of model parameter fitting problems. The new method is proved to be convergent with a global rate of decrease of the averaged gradient's norm of the form $\calO(\log(k)/\sqrt{k+1})$. Numerical experiments on several applications show that it is competitive. |
| title | prunAdag: an adaptive pruning-aware gradient method |
| topic | Optimization and Control 49M05, 49M15, 65K10, 68Q25, 90C26 F.2.1; I.2.6; G.1.6 |
| url | https://arxiv.org/abs/2502.08308 |