Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.28591 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915899903049728 |
|---|---|
| author | Kuehn, Christian Kuntz, Sara-Viola Wöhrer, Tobias |
| author_facet | Kuehn, Christian Kuntz, Sara-Viola Wöhrer, Tobias |
| contents | We analyze the universal approximation constraints of narrow Residual Neural Networks (ResNets) both theoretically and numerically. For deep neural networks without input space augmentation, a central constraint is the inability to represent critical points of the input-output map. We prove that this has global consequences for target function approximations and show that the manifestation of this defect is typically a shift of the critical point to infinity, which we call the ``tunnel effect'' in the context of classification tasks. While ResNets offer greater expressivity than standard multilayer perceptrons (MLPs), their capability strongly depends on the signal ratio between the skip and residual channels. We establish quantitative approximation bounds for both the residual-dominant (close to MLP) and skip-dominant (close to neural ODE) regimes. These estimates depend explicitly on the channel ratio and uniform network weight bounds. Low-dimensional examples further provide a detailed analysis of the different ResNet regimes and how architecture-target incompatibility influences the approximation error. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_28591 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Universal Approximation Constraints of Narrow ResNets: The Tunnel Effect Kuehn, Christian Kuntz, Sara-Viola Wöhrer, Tobias Dynamical Systems Machine Learning 41A30, 58K05, 68T07 We analyze the universal approximation constraints of narrow Residual Neural Networks (ResNets) both theoretically and numerically. For deep neural networks without input space augmentation, a central constraint is the inability to represent critical points of the input-output map. We prove that this has global consequences for target function approximations and show that the manifestation of this defect is typically a shift of the critical point to infinity, which we call the ``tunnel effect'' in the context of classification tasks. While ResNets offer greater expressivity than standard multilayer perceptrons (MLPs), their capability strongly depends on the signal ratio between the skip and residual channels. We establish quantitative approximation bounds for both the residual-dominant (close to MLP) and skip-dominant (close to neural ODE) regimes. These estimates depend explicitly on the channel ratio and uniform network weight bounds. Low-dimensional examples further provide a detailed analysis of the different ResNet regimes and how architecture-target incompatibility influences the approximation error. |
| title | Universal Approximation Constraints of Narrow ResNets: The Tunnel Effect |
| topic | Dynamical Systems Machine Learning 41A30, 58K05, 68T07 |
| url | https://arxiv.org/abs/2603.28591 |