Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2305.19082 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917557085143040 |
|---|---|
| author | Wu, Lei |
| author_facet | Wu, Lei |
| contents | An important problem in machine learning theory is to understand the approximation and generalization properties of two-layer neural networks in high dimensions. To this end, researchers have introduced the Barron space $\mathcal{B}_s(Ω)$ and the spectral Barron space $\mathcal{F}_s(Ω)$, where the index $s\in [0,\infty)$ indicates the smoothness of functions within these spaces and $Ω\subset\mathbb{R}^d$ denotes the input domain. However, the precise relationship between the two types of Barron spaces remains unclear. In this paper, we establish a continuous embedding between them as implied by the following inequality: for any $δ\in (0,1), s\in \mathbb{N}^{+}$ and $f: Ω\mapsto\mathbb{R}$, it holds that \[ δ\|f\|_{\mathcal{F}_{s-δ}(Ω)}\lesssim_s \|f\|_{\mathcal{B}_s(Ω)}\lesssim_s \|f\|_{\mathcal{F}_{s+1}(Ω)}. \]
Importantly, the constants do not depend on the input dimension $d$, suggesting that the embedding is effective in high dimensions. Moreover, we also show that the lower and upper bound are both tight. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_19082 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Embedding Inequalities for Barron-type Spaces Wu, Lei Machine Learning Numerical Analysis 68T07 An important problem in machine learning theory is to understand the approximation and generalization properties of two-layer neural networks in high dimensions. To this end, researchers have introduced the Barron space $\mathcal{B}_s(Ω)$ and the spectral Barron space $\mathcal{F}_s(Ω)$, where the index $s\in [0,\infty)$ indicates the smoothness of functions within these spaces and $Ω\subset\mathbb{R}^d$ denotes the input domain. However, the precise relationship between the two types of Barron spaces remains unclear. In this paper, we establish a continuous embedding between them as implied by the following inequality: for any $δ\in (0,1), s\in \mathbb{N}^{+}$ and $f: Ω\mapsto\mathbb{R}$, it holds that \[ δ\|f\|_{\mathcal{F}_{s-δ}(Ω)}\lesssim_s \|f\|_{\mathcal{B}_s(Ω)}\lesssim_s \|f\|_{\mathcal{F}_{s+1}(Ω)}. \] Importantly, the constants do not depend on the input dimension $d$, suggesting that the embedding is effective in high dimensions. Moreover, we also show that the lower and upper bound are both tight. |
| title | Embedding Inequalities for Barron-type Spaces |
| topic | Machine Learning Numerical Analysis 68T07 |
| url | https://arxiv.org/abs/2305.19082 |