Exact Dual-Space Optimization for Neural Networks
Fuente:
Zenodo
Enregistré dans:
| Auteur principal: | |
|---|---|
| Format: | Recurso digital |
| Langue: | anglais |
| Publié: |
Zenodo
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866902237317431296 |
|---|---|
| author | Perfilev, Vladimir |
| author_facet | Perfilev, Vladimir |
| contents | <p>We propose shifting neural network optimization from the parameter space into a dual space of dimensionality. Our core mechanism is the M-matrix—an analytical per-layer Jacobian derived from forward-pass activations, yielding exact gradients without a global computational graph. In this dual formulation, every weight update is the exact optimum of a least-squares problem, solved via a matrix-free Conjugate Gradient (CG) method.<br><br>Code: <a title="Exact Dual-Space Solver" href="https://github.com/v-perfilev/exact-dual-space-solver" rel="noopener">GitHub</a></p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_19338035 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Exact Dual-Space Optimization for Neural Networks Perfilev, Vladimir Dual-space optimization M-chain BCG FCG M-grad Layer parallel architecture <p>We propose shifting neural network optimization from the parameter space into a dual space of dimensionality. Our core mechanism is the M-matrix—an analytical per-layer Jacobian derived from forward-pass activations, yielding exact gradients without a global computational graph. In this dual formulation, every weight update is the exact optimum of a least-squares problem, solved via a matrix-free Conjugate Gradient (CG) method.<br><br>Code: <a title="Exact Dual-Space Solver" href="https://github.com/v-perfilev/exact-dual-space-solver" rel="noopener">GitHub</a></p> |
| title | Exact Dual-Space Optimization for Neural Networks |
| topic | Dual-space optimization M-chain BCG FCG M-grad Layer parallel architecture |
| url | https://doi.org/10.5281/zenodo.19338035 |