Deep Neural Networks with General Activations: Super-Convergence in Sobolev Norms
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918402560360448 |
|---|---|
| author | Yang, Yahong He, Juncai |
| author_facet | Yang, Yahong He, Juncai |
| contents | This paper establishes a comprehensive approximation result for deep fully-connected neural networks with commonly-used and general activation functions in Sobolev spaces $W^{n,\infty}$, with errors measured in the $W^{m,p}$-norm for $m < n$ and $1\le p \le \infty$. The derived rates surpass those of classical numerical approximation techniques, such as finite element and spectral methods, exhibiting a phenomenon we refer to as \emph{super-convergence}. Our analysis shows that deep networks with general activations can approximate weak solutions of partial differential equations (PDEs) with superior accuracy compared to traditional numerical methods at the approximation level. Furthermore, this work closes a significant gap in the error-estimation theory for neural-network-based approaches to PDEs, offering a unified theoretical foundation for their use in scientific computing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_05141 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Deep Neural Networks with General Activations: Super-Convergence in Sobolev Norms Yang, Yahong He, Juncai Machine Learning Numerical Analysis 68T07, 41A30, 35Q68 F.1.1; G.1.2; I.2.6 This paper establishes a comprehensive approximation result for deep fully-connected neural networks with commonly-used and general activation functions in Sobolev spaces $W^{n,\infty}$, with errors measured in the $W^{m,p}$-norm for $m < n$ and $1\le p \le \infty$. The derived rates surpass those of classical numerical approximation techniques, such as finite element and spectral methods, exhibiting a phenomenon we refer to as \emph{super-convergence}. Our analysis shows that deep networks with general activations can approximate weak solutions of partial differential equations (PDEs) with superior accuracy compared to traditional numerical methods at the approximation level. Furthermore, this work closes a significant gap in the error-estimation theory for neural-network-based approaches to PDEs, offering a unified theoretical foundation for their use in scientific computing. |
| title | Deep Neural Networks with General Activations: Super-Convergence in Sobolev Norms |
| topic | Machine Learning Numerical Analysis 68T07, 41A30, 35Q68 F.1.1; G.1.2; I.2.6 |
| url | https://arxiv.org/abs/2508.05141 |