Geometry of Singular Foliations and Learning Manifolds in ReLU Networks via the Data Information Matrix
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917333946073088 |
|---|---|
| author | Tron, Eliot Fioresi, Rita |
| author_facet | Tron, Eliot Fioresi, Rita |
| contents | Understanding how real data is distributed in high dimensional spaces is the key to many tasks in machine learning. We want to provide a natural geometric structure on the space of data employing a ReLU neural network trained as a classifier. Through the Data Information Matrix (DIM), a variation of the Fisher information matrix, the model will discern a singular foliation structure on the space of data. We show that the singular points of such foliation are contained in a measure zero set, and that a local regular foliation exists almost everywhere. Experiments show that the data is correlated with leaves of such foliation. Moreover we show the potential of our approach for knowledge transfer by analyzing the spectrum of the DIM to measure distances between datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_07412 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Geometry of Singular Foliations and Learning Manifolds in ReLU Networks via the Data Information Matrix Tron, Eliot Fioresi, Rita Machine Learning Understanding how real data is distributed in high dimensional spaces is the key to many tasks in machine learning. We want to provide a natural geometric structure on the space of data employing a ReLU neural network trained as a classifier. Through the Data Information Matrix (DIM), a variation of the Fisher information matrix, the model will discern a singular foliation structure on the space of data. We show that the singular points of such foliation are contained in a measure zero set, and that a local regular foliation exists almost everywhere. Experiments show that the data is correlated with leaves of such foliation. Moreover we show the potential of our approach for knowledge transfer by analyzing the spectrum of the DIM to measure distances between datasets. |
| title | Geometry of Singular Foliations and Learning Manifolds in ReLU Networks via the Data Information Matrix |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2409.07412 |