Least Squares Regression Can Exhibit Under-Parameterized Double Descent
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910665540632576 |
|---|---|
| author | Li, Xinyue Sonthalia, Rishi |
| author_facet | Li, Xinyue Sonthalia, Rishi |
| contents | The relationship between the number of training data points, the number of parameters, and the generalization capabilities of models has been widely studied. Previous work has shown that double descent can occur in the over-parameterized regime and that the standard bias-variance trade-off holds in the under-parameterized regime. These works provide multiple reasons for the existence of the peak. We postulate that the location of the peak depends on the technical properties of both the spectrum as well as the eigenvectors of the sample covariance. We present two simple examples that provably exhibit double descent in the under-parameterized regime and do not seem to occur for reasons provided in prior work. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_14689 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Least Squares Regression Can Exhibit Under-Parameterized Double Descent Li, Xinyue Sonthalia, Rishi Machine Learning Statistics Theory The relationship between the number of training data points, the number of parameters, and the generalization capabilities of models has been widely studied. Previous work has shown that double descent can occur in the over-parameterized regime and that the standard bias-variance trade-off holds in the under-parameterized regime. These works provide multiple reasons for the existence of the peak. We postulate that the location of the peak depends on the technical properties of both the spectrum as well as the eigenvectors of the sample covariance. We present two simple examples that provably exhibit double descent in the under-parameterized regime and do not seem to occur for reasons provided in prior work. |
| title | Least Squares Regression Can Exhibit Under-Parameterized Double Descent |
| topic | Machine Learning Statistics Theory |
| url | https://arxiv.org/abs/2305.14689 |