Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915095263576064 |
|---|---|
| author | Chen, Ziang Ge, Rong |
| author_facet | Chen, Ziang Ge, Rong |
| contents | In this work, we study the mean-field flow for learning subspace-sparse polynomials using stochastic gradient descent and two-layer neural networks, where the input distribution is standard Gaussian and the output only depends on the projection of the input onto a low-dimensional subspace. We establish a necessary condition for SGD-learnability, involving both the characteristics of the target function and the expressiveness of the activation function. In addition, we prove that the condition is almost sufficient, in the sense that a condition slightly stronger than the necessary condition can guarantee the exponential decay of the loss functional to zero. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_08948 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input Chen, Ziang Ge, Rong Machine Learning Analysis of PDEs In this work, we study the mean-field flow for learning subspace-sparse polynomials using stochastic gradient descent and two-layer neural networks, where the input distribution is standard Gaussian and the output only depends on the projection of the input onto a low-dimensional subspace. We establish a necessary condition for SGD-learnability, involving both the characteristics of the target function and the expressiveness of the activation function. In addition, we prove that the condition is almost sufficient, in the sense that a condition slightly stronger than the necessary condition can guarantee the exponential decay of the loss functional to zero. |
| title | Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input |
| topic | Machine Learning Analysis of PDEs |
| url | https://arxiv.org/abs/2402.08948 |