Deep Networks are Reproducing Kernel Chains
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929663517917184 |
|---|---|
| author | Heeringa, Tjeerd Jan Spek, Len Brune, Christoph |
| author_facet | Heeringa, Tjeerd Jan Spek, Len Brune, Christoph |
| contents | Identifying an appropriate function space for deep neural networks remains a key open question. While shallow neural networks are naturally associated with Reproducing Kernel Banach Spaces (RKBS), deep networks present unique challenges. In this work, we extend RKBS to chain RKBS (cRKBS), a new framework that composes kernels rather than functions, preserving the desirable properties of RKBS. We prove that any deep neural network function is a neural cRKBS function, and conversely, any neural cRKBS function defined on a finite dataset corresponds to a deep neural network. This approach provides a sparse solution to the empirical risk minimization problem, requiring no more than $N$ neurons per layer, where $N$ is the number of data points. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_03697 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Deep Networks are Reproducing Kernel Chains Heeringa, Tjeerd Jan Spek, Len Brune, Christoph Machine Learning Functional Analysis 46E15 (Primary) 46B10, 68T07, 26B40 (Secondary) I.2.6; G.1.6 Identifying an appropriate function space for deep neural networks remains a key open question. While shallow neural networks are naturally associated with Reproducing Kernel Banach Spaces (RKBS), deep networks present unique challenges. In this work, we extend RKBS to chain RKBS (cRKBS), a new framework that composes kernels rather than functions, preserving the desirable properties of RKBS. We prove that any deep neural network function is a neural cRKBS function, and conversely, any neural cRKBS function defined on a finite dataset corresponds to a deep neural network. This approach provides a sparse solution to the empirical risk minimization problem, requiring no more than $N$ neurons per layer, where $N$ is the number of data points. |
| title | Deep Networks are Reproducing Kernel Chains |
| topic | Machine Learning Functional Analysis 46E15 (Primary) 46B10, 68T07, 26B40 (Secondary) I.2.6; G.1.6 |
| url | https://arxiv.org/abs/2501.03697 |