Large Deviations of Gaussian Neural Networks with ReLU activation
Fuente:
arXiv
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911432443953152 |
|---|---|
| author | Vogel, Quirin |
| author_facet | Vogel, Quirin |
| contents | We prove a large deviation principle for deep neural networks with Gaussian weights and at most linearly growing activation functions, such as ReLU. This generalises earlier work, in which bounded and continuous activation functions were considered. In practice, linearly growing activation functions such as ReLU are most commonly used. We furthermore simplify previous expressions for the rate function and provide a power-series expansions for the ReLU case. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_16958 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Large Deviations of Gaussian Neural Networks with ReLU activation Vogel, Quirin Machine Learning Probability 60F10, 68T07 We prove a large deviation principle for deep neural networks with Gaussian weights and at most linearly growing activation functions, such as ReLU. This generalises earlier work, in which bounded and continuous activation functions were considered. In practice, linearly growing activation functions such as ReLU are most commonly used. We furthermore simplify previous expressions for the rate function and provide a power-series expansions for the ReLU case. |
| title | Large Deviations of Gaussian Neural Networks with ReLU activation |
| topic | Machine Learning Probability 60F10, 68T07 |
| url | https://arxiv.org/abs/2405.16958 |