Approximation Error and Complexity Bounds for ReLU Networks on Low-Regular Function Spaces
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915816820178944 |
|---|---|
| author | Davis, Owen Geraci, Gianluca Motamed, Mohammad |
| author_facet | Davis, Owen Geraci, Gianluca Motamed, Mohammad |
| contents | In this work, we consider the approximation of a large class of bounded functions, with minimal regularity assumptions, by ReLU neural networks. We show that the approximation error can be bounded from above by a quantity proportional to the uniform norm of the target function and inversely proportional to the product of network width and depth. We inherit this approximation error bound from Fourier features residual networks, a type of neural network that uses complex exponential activation functions. Our proof is constructive and proceeds by conducting a careful complexity analysis associated with the approximation of a Fourier features residual network by a ReLU network. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_06727 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Approximation Error and Complexity Bounds for ReLU Networks on Low-Regular Function Spaces Davis, Owen Geraci, Gianluca Motamed, Mohammad Machine Learning 41A25, 41A30, 41A46, 68T07 In this work, we consider the approximation of a large class of bounded functions, with minimal regularity assumptions, by ReLU neural networks. We show that the approximation error can be bounded from above by a quantity proportional to the uniform norm of the target function and inversely proportional to the product of network width and depth. We inherit this approximation error bound from Fourier features residual networks, a type of neural network that uses complex exponential activation functions. Our proof is constructive and proceeds by conducting a careful complexity analysis associated with the approximation of a Fourier features residual network by a ReLU network. |
| title | Approximation Error and Complexity Bounds for ReLU Networks on Low-Regular Function Spaces |
| topic | Machine Learning 41A25, 41A30, 41A46, 68T07 |
| url | https://arxiv.org/abs/2405.06727 |