Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2306.01992 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910318142160896 |
|---|---|
| author | Sellke, Mark |
| author_facet | Sellke, Mark |
| contents | We study the sample complexity of learning ReLU neural networks from the point of view of generalization. Given norm constraints on the weight matrices, a common approach is to estimate the Rademacher complexity of the associated function class. Previously Golowich-Rakhlin-Shamir (2020) obtained a bound independent of the network size (scaling with a product of Frobenius norms) except for a factor of the square-root depth. We give a refinement which often has no explicit depth-dependence at all. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_01992 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | On Size-Independent Sample Complexity of ReLU Networks Sellke, Mark Machine Learning We study the sample complexity of learning ReLU neural networks from the point of view of generalization. Given norm constraints on the weight matrices, a common approach is to estimate the Rademacher complexity of the associated function class. Previously Golowich-Rakhlin-Shamir (2020) obtained a bound independent of the network size (scaling with a product of Frobenius norms) except for a factor of the square-root depth. We give a refinement which often has no explicit depth-dependence at all. |
| title | On Size-Independent Sample Complexity of ReLU Networks |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2306.01992 |