Saved in:
| Main Authors: | , |
|---|---|
| Format: | Recurso digital |
| Language: | |
| Published: |
Zenodo
2025
|
| Online Access: | https://doi.org/10.5281/zenodo.17834155 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Table of Contents:
- Deep learning models have achieved remarkable success across various domains, yet a comprehensive theoretical understanding of their generalization capabilities, particularly in the overparameterized regime, remains an active area of research. This paper explores the concept of the intrinsic kernel dimension as a fundamental property influencing the generalization performance of deep neural networks. By drawing parallels between deep learning architectures and kernel methods, we hypothesize that the effective dimensionality of the feature space induced by deep networks, often much lower than the ambient dimension, dictates their ability to generalize well to unseen data. We review established theories of generalization and recent advances linking deep learning to kernel methods, such as the Neural Tangent Kernel (NTK). We discuss how techniques like multi-kernel approximation and low-rank tensor decomposition, traditionally used to enhance generalization and reduce complexity in kernel-based and deep models, respectively, implicitly or explicitly control this intrinsic dimension. Our methodology outlines approaches to quantify this dimension through analyses of network activations and effective kernel matrices. We argue that understanding and controlling the intrinsic kernel dimension offers new avenues for designing more robust and efficient deep learning models, providing insights into the "why" behind deep learning's empirical successes and theoretical challenges.