Barcodes as Summary of Loss Function Topology
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2019
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912184467980288 |
|---|---|
| author | Barannikov, Serguei Korotin, Alexander Oganesyan, Dmitry Emtsev, Daniil Burnaev, Evgeny |
| author_facet | Barannikov, Serguei Korotin, Alexander Oganesyan, Dmitry Emtsev, Daniil Burnaev, Evgeny |
| contents | We propose to study neural networks' loss surfaces by methods of topological data analysis. We suggest to apply barcodes of Morse complexes to explore topology of loss surfaces. An algorithm for calculations of the loss function's barcodes of local minima is described. We have conducted experiments for calculating barcodes of local minima for benchmark functions and for loss surfaces of small neural networks. Our experiments confirm our two principal observations for neural networks' loss surfaces. First, the barcodes of local minima are located in a small lower part of the range of values of neural networks' loss function. Secondly, increase of the neural network's depth and width lowers the barcodes of local minima. This has some natural implications for the neural network's learning and for its generalization properties. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_1912_00043 |
| institution | arXiv |
| publishDate | 2019 |
| record_format | arxiv |
| spellingShingle | Barcodes as Summary of Loss Function Topology Barannikov, Serguei Korotin, Alexander Oganesyan, Dmitry Emtsev, Daniil Burnaev, Evgeny Machine Learning Algebraic Topology Dynamical Systems Optimization and Control We propose to study neural networks' loss surfaces by methods of topological data analysis. We suggest to apply barcodes of Morse complexes to explore topology of loss surfaces. An algorithm for calculations of the loss function's barcodes of local minima is described. We have conducted experiments for calculating barcodes of local minima for benchmark functions and for loss surfaces of small neural networks. Our experiments confirm our two principal observations for neural networks' loss surfaces. First, the barcodes of local minima are located in a small lower part of the range of values of neural networks' loss function. Secondly, increase of the neural network's depth and width lowers the barcodes of local minima. This has some natural implications for the neural network's learning and for its generalization properties. |
| title | Barcodes as Summary of Loss Function Topology |
| topic | Machine Learning Algebraic Topology Dynamical Systems Optimization and Control |
| url | https://arxiv.org/abs/1912.00043 |