Sharp Risk Bounds for Early-Stopping in Gaussian Linear Regression
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917441089568768 |
|---|---|
| author | Wegel, Tobias Kur, Gil Rebeschini, Patrick |
| author_facet | Wegel, Tobias Kur, Gil Rebeschini, Patrick |
| contents | We study early-stopped mirror descent (ESMD) for high-dimensional Gaussian linear regression over arbitrary convex bodies and design matrices, where the task is to minimize the in-sample mean squared error. Our main result shows that some of the sharpest risk bounds for the least squares estimator (LSE), based on the local Gaussian width, extend to ESMD. We derive sufficient conditions on the potential, expressed via the Minkowski functional, under which our result holds. These conditions allow us to construct new potentials and analyze existing ones. Our results then yield general sufficient conditions for minimax optimality of ESMD, provide a systematic comparison with the LSE, and establish the tightest known risk bound in the $\ell_1$-constrained setting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_03426 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Sharp Risk Bounds for Early-Stopping in Gaussian Linear Regression Wegel, Tobias Kur, Gil Rebeschini, Patrick Machine Learning Statistics Theory We study early-stopped mirror descent (ESMD) for high-dimensional Gaussian linear regression over arbitrary convex bodies and design matrices, where the task is to minimize the in-sample mean squared error. Our main result shows that some of the sharpest risk bounds for the least squares estimator (LSE), based on the local Gaussian width, extend to ESMD. We derive sufficient conditions on the potential, expressed via the Minkowski functional, under which our result holds. These conditions allow us to construct new potentials and analyze existing ones. Our results then yield general sufficient conditions for minimax optimality of ESMD, provide a systematic comparison with the LSE, and establish the tightest known risk bound in the $\ell_1$-constrained setting. |
| title | Sharp Risk Bounds for Early-Stopping in Gaussian Linear Regression |
| topic | Machine Learning Statistics Theory |
| url | https://arxiv.org/abs/2503.03426 |