Asymptotic Optimism of Random-Design Linear and Kernel Regression Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908491246993408 |
|---|---|
| author | Luo, Hengrui Zhu, Yunzhang |
| author_facet | Luo, Hengrui Zhu, Yunzhang |
| contents | We derived the closed-form asymptotic optimism of linear regression models under random designs, and generalizes it to kernel ridge regression. Using scaled asymptotic optimism as a generic predictive model complexity measure, we studied the fundamental different behaviors of linear regression model, tangent kernel (NTK) regression model and three-layer fully connected neural networks (NN). Our contribution is two-fold: we provided theoretical ground for using scaled optimism as a model predictive complexity measure; and we show empirically that NN with ReLUs behaves differently from kernel models under this measure. With resampling techniques, we can also compute the optimism for regression models with real data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_12999 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Asymptotic Optimism of Random-Design Linear and Kernel Regression Models Luo, Hengrui Zhu, Yunzhang Machine Learning Statistics Theory 68T05, 68Q32 We derived the closed-form asymptotic optimism of linear regression models under random designs, and generalizes it to kernel ridge regression. Using scaled asymptotic optimism as a generic predictive model complexity measure, we studied the fundamental different behaviors of linear regression model, tangent kernel (NTK) regression model and three-layer fully connected neural networks (NN). Our contribution is two-fold: we provided theoretical ground for using scaled optimism as a model predictive complexity measure; and we show empirically that NN with ReLUs behaves differently from kernel models under this measure. With resampling techniques, we can also compute the optimism for regression models with real data. |
| title | Asymptotic Optimism of Random-Design Linear and Kernel Regression Models |
| topic | Machine Learning Statistics Theory 68T05, 68Q32 |
| url | https://arxiv.org/abs/2502.12999 |