A Non-Monotone Preconditioned Trust-Region Method for Neural Network Training
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909043827671040 |
|---|---|
| author | Angino, Andrea Çapriqi, Bindi Likaj, Shega Trotti, Ken Krause, Rolf |
| author_facet | Angino, Andrea Çapriqi, Bindi Likaj, Shega Trotti, Ken Krause, Rolf |
| contents | Training deep neural networks at scale can benefit from domain decomposition, where the network is split into subdomains trained in parallel and coupled by a global trust-region mechanism. Building on the Additively Preconditioned Trust-Region Strategy (APTS), we propose a non-monotone variant with a nonlinear additive Schwarz preconditioner that combines parallel subdomain corrections with global coarse-space directions. A windowed acceptance criterion allows controlled objective increases, avoiding needless rejection of effective coarse steps. The resulting non-monotone APTS (NAPTS) preserves accuracy while reducing CPU time by 30\% and cutting rejected steps to one third of those in APTS. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_14860 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A Non-Monotone Preconditioned Trust-Region Method for Neural Network Training Angino, Andrea Çapriqi, Bindi Likaj, Shega Trotti, Ken Krause, Rolf Optimization and Control Machine Learning Training deep neural networks at scale can benefit from domain decomposition, where the network is split into subdomains trained in parallel and coupled by a global trust-region mechanism. Building on the Additively Preconditioned Trust-Region Strategy (APTS), we propose a non-monotone variant with a nonlinear additive Schwarz preconditioner that combines parallel subdomain corrections with global coarse-space directions. A windowed acceptance criterion allows controlled objective increases, avoiding needless rejection of effective coarse steps. The resulting non-monotone APTS (NAPTS) preserves accuracy while reducing CPU time by 30\% and cutting rejected steps to one third of those in APTS. |
| title | A Non-Monotone Preconditioned Trust-Region Method for Neural Network Training |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2605.14860 |