Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914366788468736 |
|---|---|
| author | Balasubramanian, Krishnakumar Ross, Nathan |
| author_facet | Balasubramanian, Krishnakumar Ross, Nathan |
| contents | We study the Finite-Dimensional Distributions (FDDs) of deep neural networks with randomly initialized weights that have finite-order moments. Specifically, we establish Gaussian approximation bounds in the Wasserstein-$1$ norm between the FDDs and their Gaussian limit assuming a Lipschitz activation function and allowing the layer widths to grow to infinity at arbitrary relative rates. In the special case where all widths are proportional to a common scale parameter $n$ and there are $L-1$ hidden layers, we obtain convergence rates of order $n^{-({1}/{6})^{L-1} + ε}$, for any $ε> 0$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_12686 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights Balasubramanian, Krishnakumar Ross, Nathan Machine Learning Probability Statistics Theory We study the Finite-Dimensional Distributions (FDDs) of deep neural networks with randomly initialized weights that have finite-order moments. Specifically, we establish Gaussian approximation bounds in the Wasserstein-$1$ norm between the FDDs and their Gaussian limit assuming a Lipschitz activation function and allowing the layer widths to grow to infinity at arbitrary relative rates. In the special case where all widths are proportional to a common scale parameter $n$ and there are $L-1$ hidden layers, we obtain convergence rates of order $n^{-({1}/{6})^{L-1} + ε}$, for any $ε> 0$. |
| title | Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights |
| topic | Machine Learning Probability Statistics Theory |
| url | https://arxiv.org/abs/2507.12686 |