On Functional Dimension and Persistent Pseudodimension
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910729415688192 |
|---|---|
| author | Grigsby, J. Elisenda Lindsey, Kathryn |
| author_facet | Grigsby, J. Elisenda Lindsey, Kathryn |
| contents | For any fixed feedforward ReLU neural network architecture, it is well-known that many different parameter settings can determine the same function. It is less well-known that the degree of this redundancy is inhomogeneous across parameter space. In this work, we discuss two locally applicable complexity measures for ReLU network classes and what we know about the relationship between them: (1) the local functional dimension [14, 18], and (2) a local version of VC dimension that we call persistent pseudodimension. The former is easy to compute on finite batches of points; the latter should give local bounds on the generalization gap, which would inform an understanding of the mechanics of the double descent phenomenon [7]. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_17191 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On Functional Dimension and Persistent Pseudodimension Grigsby, J. Elisenda Lindsey, Kathryn Machine Learning Algebraic Geometry Combinatorics 57R70, 57Q99, 52B70, 52C35 For any fixed feedforward ReLU neural network architecture, it is well-known that many different parameter settings can determine the same function. It is less well-known that the degree of this redundancy is inhomogeneous across parameter space. In this work, we discuss two locally applicable complexity measures for ReLU network classes and what we know about the relationship between them: (1) the local functional dimension [14, 18], and (2) a local version of VC dimension that we call persistent pseudodimension. The former is easy to compute on finite batches of points; the latter should give local bounds on the generalization gap, which would inform an understanding of the mechanics of the double descent phenomenon [7]. |
| title | On Functional Dimension and Persistent Pseudodimension |
| topic | Machine Learning Algebraic Geometry Combinatorics 57R70, 57Q99, 52B70, 52C35 |
| url | https://arxiv.org/abs/2410.17191 |