The geometry of the deep linear network
Fuente:
arXiv
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917837251018752 |
|---|---|
| author | Menon, Govind |
| author_facet | Menon, Govind |
| contents | This article provides an expository account of training dynamics in the Deep Linear Network (DLN) from the perspective of the geometric theory of dynamical systems. Rigorous results by several authors are unified into a thermodynamic framework for deep learning.
The analysis begins with a characterization of the invariant manifolds and Riemannian geometry in the DLN. This is followed by exact formulas for a Boltzmann entropy, as well as stochastic gradient descent of free energy using a Riemannian Langevin Equation. Several links between the DLN and other areas of mathematics are discussed, along with some open questions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_09004 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The geometry of the deep linear network Menon, Govind Neural and Evolutionary Computing Dynamical Systems Probability Adaptation and Self-Organizing Systems 68T07, 58D17, 37N40 This article provides an expository account of training dynamics in the Deep Linear Network (DLN) from the perspective of the geometric theory of dynamical systems. Rigorous results by several authors are unified into a thermodynamic framework for deep learning. The analysis begins with a characterization of the invariant manifolds and Riemannian geometry in the DLN. This is followed by exact formulas for a Boltzmann entropy, as well as stochastic gradient descent of free energy using a Riemannian Langevin Equation. Several links between the DLN and other areas of mathematics are discussed, along with some open questions. |
| title | The geometry of the deep linear network |
| topic | Neural and Evolutionary Computing Dynamical Systems Probability Adaptation and Self-Organizing Systems 68T07, 58D17, 37N40 |
| url | https://arxiv.org/abs/2411.09004 |