Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.06096 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915729394106368 |
|---|---|
| author | Rahimi, Ali |
| author_facet | Rahimi, Ali |
| contents | We describe an exact algorithm to solve linear systems of the form $Hx=b$ where $H$ is the Hessian of a deep net. The method computes Hessian-inverse-vector products without storing the Hessian or its inverse. It requires time and storage that scale linearly in the number of layers. This is in contrast to the naive approach of first computing the Hessian, then solving the linear system, which takes storage and time that are respectively quadratic and cubic in the number of layers. The Hessian-inverse-vector product method scales roughly like Pearlmutter's algorithm for computing Hessian-vector products. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_06096 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | The Hessian of tall-skinny networks is easy to invert Rahimi, Ali Machine Learning We describe an exact algorithm to solve linear systems of the form $Hx=b$ where $H$ is the Hessian of a deep net. The method computes Hessian-inverse-vector products without storing the Hessian or its inverse. It requires time and storage that scale linearly in the number of layers. This is in contrast to the naive approach of first computing the Hessian, then solving the linear system, which takes storage and time that are respectively quadratic and cubic in the number of layers. The Hessian-inverse-vector product method scales roughly like Pearlmutter's algorithm for computing Hessian-vector products. |
| title | The Hessian of tall-skinny networks is easy to invert |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2601.06096 |