Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.18388 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912662696230912 |
|---|---|
| author | Lu, Jian Huang, Xiaohuang |
| author_facet | Lu, Jian Huang, Xiaohuang |
| contents | This paper investigates the approximation properties of shallow neural networks with activation functions that are powers of exponential functions. It focuses on the dependence of the approximation rate on the dimension and the smoothness of the function being approximated within the Barron function space. We examine the approximation rates of ReLU$^{k}$ activation functions, proving that the optimal rate cannot be achieved under $\ell^{1}$-bounded coefficients or insufficient smoothness conditions.
We also establish optimal approximation rates in various norms for functions in Barron spaces and Sobolev spaces, confirming the curse of dimensionality. Our results clarify the limits of shallow neural networks' approximation capabilities and offer insights into the selection of activation functions and network structures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_18388 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Approximation Rates of Shallow Neural Networks: Barron Spaces, Activation Functions and Optimality Analysis Lu, Jian Huang, Xiaohuang Machine Learning 41A46 This paper investigates the approximation properties of shallow neural networks with activation functions that are powers of exponential functions. It focuses on the dependence of the approximation rate on the dimension and the smoothness of the function being approximated within the Barron function space. We examine the approximation rates of ReLU$^{k}$ activation functions, proving that the optimal rate cannot be achieved under $\ell^{1}$-bounded coefficients or insufficient smoothness conditions. We also establish optimal approximation rates in various norms for functions in Barron spaces and Sobolev spaces, confirming the curse of dimensionality. Our results clarify the limits of shallow neural networks' approximation capabilities and offer insights into the selection of activation functions and network structures. |
| title | Approximation Rates of Shallow Neural Networks: Barron Spaces, Activation Functions and Optimality Analysis |
| topic | Machine Learning 41A46 |
| url | https://arxiv.org/abs/2510.18388 |