On the expressivity of deep Heaviside networks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913813579694080 |
|---|---|
| author | Kong, Insung Chen, Juntong Langer, Sophie Schmidt-Hieber, Johannes |
| author_facet | Kong, Insung Chen, Juntong Langer, Sophie Schmidt-Hieber, Johannes |
| contents | We show that deep Heaviside networks (DHNs) have limited expressiveness but that this can be overcome by including either skip connections or neurons with linear activation. We provide lower and upper bounds for the Vapnik-Chervonenkis (VC) dimensions and approximation rates of these network classes. As an application, we derive statistical convergence rates for DHN fits in the nonparametric regression model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_00110 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On the expressivity of deep Heaviside networks Kong, Insung Chen, Juntong Langer, Sophie Schmidt-Hieber, Johannes Machine Learning Numerical Analysis We show that deep Heaviside networks (DHNs) have limited expressiveness but that this can be overcome by including either skip connections or neurons with linear activation. We provide lower and upper bounds for the Vapnik-Chervonenkis (VC) dimensions and approximation rates of these network classes. As an application, we derive statistical convergence rates for DHN fits in the nonparametric regression model. |
| title | On the expressivity of deep Heaviside networks |
| topic | Machine Learning Numerical Analysis |
| url | https://arxiv.org/abs/2505.00110 |