Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909705060745216 |
|---|---|
| author | Takeuchi, Jun'ichi Takeishi, Yoshinari Murata, Noboru Mimura, Kazushi Ho, Ka Long Keith Nagaoka, Hiroshi |
| author_facet | Takeuchi, Jun'ichi Takeishi, Yoshinari Murata, Noboru Mimura, Kazushi Ho, Ka Long Keith Nagaoka, Hiroshi |
| contents | Fisher information matrices and neural tangent kernels (NTK) for 2-layer ReLU networks with random hidden weight are argued. We discuss the relation between both notions as a linear transformation and show that spectral decomposition of NTK with concrete forms of eigenfunctions with major eigenvalues. We also obtain an approximation formula of the functions presented by the 2-layer neural networks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_18555 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights Takeuchi, Jun'ichi Takeishi, Yoshinari Murata, Noboru Mimura, Kazushi Ho, Ka Long Keith Nagaoka, Hiroshi Machine Learning Fisher information matrices and neural tangent kernels (NTK) for 2-layer ReLU networks with random hidden weight are argued. We discuss the relation between both notions as a linear transformation and show that spectral decomposition of NTK with concrete forms of eigenfunctions with major eigenvalues. We also obtain an approximation formula of the functions presented by the 2-layer neural networks. |
| title | Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2507.18555 |