Fractal and Regular Geometry of Deep Neural Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911403057610752 |
|---|---|
| author | Di Lillo, Simmaco Marinucci, Domenico Salvi, Michele Vigogna, Stefano |
| author_facet | Di Lillo, Simmaco Marinucci, Domenico Salvi, Michele Vigogna, Stefano |
| contents | We study the geometric properties of random neural networks by investigating the boundary volumes of their excursion sets for different activation functions, as the depth increases. More specifically, we show that, for activations which are not very regular (e.g., the Heaviside step function), the boundary volumes exhibit fractal behavior, with their Hausdorff dimension monotonically increasing with the depth. On the other hand, for activations which are more regular (e.g., ReLU, logistic and $\tanh$), as the depth increases, the expected boundary volumes can either converge to zero, remain constant or diverge exponentially, depending on a single spectral parameter which can be easily computed. Our theoretical results are confirmed in some numerical experiments based on Monte Carlo simulations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_06250 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Fractal and Regular Geometry of Deep Neural Networks Di Lillo, Simmaco Marinucci, Domenico Salvi, Michele Vigogna, Stefano Probability Machine Learning 60G60, 62B10, 62M45, 68T07 We study the geometric properties of random neural networks by investigating the boundary volumes of their excursion sets for different activation functions, as the depth increases. More specifically, we show that, for activations which are not very regular (e.g., the Heaviside step function), the boundary volumes exhibit fractal behavior, with their Hausdorff dimension monotonically increasing with the depth. On the other hand, for activations which are more regular (e.g., ReLU, logistic and $\tanh$), as the depth increases, the expected boundary volumes can either converge to zero, remain constant or diverge exponentially, depending on a single spectral parameter which can be easily computed. Our theoretical results are confirmed in some numerical experiments based on Monte Carlo simulations. |
| title | Fractal and Regular Geometry of Deep Neural Networks |
| topic | Probability Machine Learning 60G60, 62B10, 62M45, 68T07 |
| url | https://arxiv.org/abs/2504.06250 |