Estimation of Over-parameterized Models from an Auto-Modeling Perspective
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2022
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917859062448128 |
|---|---|
| author | Jiang, Yiran Liu, Chuanhai |
| author_facet | Jiang, Yiran Liu, Chuanhai |
| contents | From a model-building perspective, we propose a paradigm shift for fitting over-parameterized models. Philosophically, the mindset is to fit models to future observations rather than to the observed sample. Technically, given an imputation method to generate future observations, we fit over-parameterized models to these future observations by optimizing an approximation of the desired expected loss function based on its sample counterpart and an adaptive $\textit{duality function}$. The required imputation method is also developed using the same estimation technique with an adaptive $m$-out-of-$n$ bootstrap approach. We illustrate its applications with the many-normal-means problem, $n < p$ linear regression, and neural network-based image classification of MNIST digits. The numerical results demonstrate its superior performance across these diverse applications. While primarily expository, the paper conducts an in-depth investigation into the theoretical aspects of the topic. It concludes with remarks on some open problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2206_01824 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Estimation of Over-parameterized Models from an Auto-Modeling Perspective Jiang, Yiran Liu, Chuanhai Methodology Statistics Theory Applications From a model-building perspective, we propose a paradigm shift for fitting over-parameterized models. Philosophically, the mindset is to fit models to future observations rather than to the observed sample. Technically, given an imputation method to generate future observations, we fit over-parameterized models to these future observations by optimizing an approximation of the desired expected loss function based on its sample counterpart and an adaptive $\textit{duality function}$. The required imputation method is also developed using the same estimation technique with an adaptive $m$-out-of-$n$ bootstrap approach. We illustrate its applications with the many-normal-means problem, $n < p$ linear regression, and neural network-based image classification of MNIST digits. The numerical results demonstrate its superior performance across these diverse applications. While primarily expository, the paper conducts an in-depth investigation into the theoretical aspects of the topic. It concludes with remarks on some open problems. |
| title | Estimation of Over-parameterized Models from an Auto-Modeling Perspective |
| topic | Methodology Statistics Theory Applications |
| url | https://arxiv.org/abs/2206.01824 |