Mathematical Foundations of Neural Tangents and Infinite-Width Networks
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915662385905664 |
|---|---|
| author | Mysore, Rachana Girish, Preksha Jayaram, Kavitha Kumar, Shrey Girish, Preksha Bagal, Shravan Sanjeev Jayaram, Kavitha Shastry, Shreya Aravind |
| author_facet | Mysore, Rachana Girish, Preksha Jayaram, Kavitha Kumar, Shrey Girish, Preksha Bagal, Shravan Sanjeev Jayaram, Kavitha Shastry, Shreya Aravind |
| contents | We investigate the mathematical foundations of neural networks in the infinite-width regime through the Neural Tangent Kernel (NTK). We propose the NTK-Eigenvalue-Controlled Residual Network (NTK-ECRN), an architecture integrating Fourier feature embeddings, residual connections with layerwise scaling, and stochastic depth to enable rigorous analysis of kernel evolution during training. Our theoretical contributions include deriving bounds on NTK dynamics, characterizing eigenvalue evolution, and linking spectral properties to generalization and optimization stability. Empirical results on synthetic and benchmark datasets validate the predicted kernel behavior and demonstrate improved training stability and generalization. This work provides a comprehensive framework bridging infinite-width theory and practical deep-learning architectures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_08264 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mathematical Foundations of Neural Tangents and Infinite-Width Networks Mysore, Rachana Girish, Preksha Jayaram, Kavitha Kumar, Shrey Girish, Preksha Bagal, Shravan Sanjeev Jayaram, Kavitha Shastry, Shreya Aravind Machine Learning 68T07, 68T05, 60B20 I.2.6; I.5.1; G.3 We investigate the mathematical foundations of neural networks in the infinite-width regime through the Neural Tangent Kernel (NTK). We propose the NTK-Eigenvalue-Controlled Residual Network (NTK-ECRN), an architecture integrating Fourier feature embeddings, residual connections with layerwise scaling, and stochastic depth to enable rigorous analysis of kernel evolution during training. Our theoretical contributions include deriving bounds on NTK dynamics, characterizing eigenvalue evolution, and linking spectral properties to generalization and optimization stability. Empirical results on synthetic and benchmark datasets validate the predicted kernel behavior and demonstrate improved training stability and generalization. This work provides a comprehensive framework bridging infinite-width theory and practical deep-learning architectures. |
| title | Mathematical Foundations of Neural Tangents and Infinite-Width Networks |
| topic | Machine Learning 68T07, 68T05, 60B20 I.2.6; I.5.1; G.3 |
| url | https://arxiv.org/abs/2512.08264 |