The Intrinsic Kernel Dimension of Deep Learning Generalization
Fuente:
Zenodo
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Recurso digital |
| Publicado: |
Zenodo
2025
|
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866901046236807168 |
|---|---|
| author | Revista, Zen MATH, 10 |
| author_facet | Revista, Zen MATH, 10 |
| contents | Deep learning models have achieved remarkable success across various domains, yet a comprehensive theoretical understanding of their generalization capabilities, particularly in the overparameterized regime, remains an active area of research. This paper explores the concept of the intrinsic kernel dimension as a fundamental property influencing the generalization performance of deep neural networks. By drawing parallels between deep learning architectures and kernel methods, we hypothesize that the effective dimensionality of the feature space induced by deep networks, often much lower than the ambient dimension, dictates their ability to generalize well to unseen data. We review established theories of generalization and recent advances linking deep learning to kernel methods, such as the Neural Tangent Kernel (NTK). We discuss how techniques like multi-kernel approximation and low-rank tensor decomposition, traditionally used to enhance generalization and reduce complexity in kernel-based and deep models, respectively, implicitly or explicitly control this intrinsic dimension. Our methodology outlines approaches to quantify this dimension through analyses of network activations and effective kernel matrices. We argue that understanding and controlling the intrinsic kernel dimension offers new avenues for designing more robust and efficient deep learning models, providing insights into the "why" behind deep learning's empirical successes and theoretical challenges. |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_17834155 |
| institution | Zenodo |
| language | |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | The Intrinsic Kernel Dimension of Deep Learning Generalization Revista, Zen MATH, 10 Deep learning models have achieved remarkable success across various domains, yet a comprehensive theoretical understanding of their generalization capabilities, particularly in the overparameterized regime, remains an active area of research. This paper explores the concept of the intrinsic kernel dimension as a fundamental property influencing the generalization performance of deep neural networks. By drawing parallels between deep learning architectures and kernel methods, we hypothesize that the effective dimensionality of the feature space induced by deep networks, often much lower than the ambient dimension, dictates their ability to generalize well to unseen data. We review established theories of generalization and recent advances linking deep learning to kernel methods, such as the Neural Tangent Kernel (NTK). We discuss how techniques like multi-kernel approximation and low-rank tensor decomposition, traditionally used to enhance generalization and reduce complexity in kernel-based and deep models, respectively, implicitly or explicitly control this intrinsic dimension. Our methodology outlines approaches to quantify this dimension through analyses of network activations and effective kernel matrices. We argue that understanding and controlling the intrinsic kernel dimension offers new avenues for designing more robust and efficient deep learning models, providing insights into the "why" behind deep learning's empirical successes and theoretical challenges. |
| title | The Intrinsic Kernel Dimension of Deep Learning Generalization |
| url | https://doi.org/10.5281/zenodo.17834155 |