Excess Mean Squared Error of Empirical Bayes Estimators
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917958191677440 |
|---|---|
| author | Ju, Yue Wahlberg, Bo Hjalmarsson, Håkan |
| author_facet | Ju, Yue Wahlberg, Bo Hjalmarsson, Håkan |
| contents | Empirical Bayes estimators are based on minimizing the average risk with the hyper-parameters in the weighting function being estimated from observed data. The performance of an empirical Bayes estimator is typically evaluated by its mean squared error (MSE). However, the explicit expression for its MSE is generally unavailable for finite sample sizes. To address this issue, we define a high-order analytical criterion: the excess MSE. It quantifies the performance difference between the maximum likelihood and empirical Bayes estimators. An explicit expression for the excess MSE of an empirical Bayes estimator employing a general data-dependent hyper-parameter estimator is derived. As specific instances, we provide excess MSE expressions for kernel-based regularized estimators using the scaled empirical Bayes, Stein unbiased risk estimation, and generalized cross-validation hyper-parameter estimators. Moreover, we propose a modification to the excess MSE expressions for regularized estimators for moderate sample sizes and show its improvement on accuracy in numerical simulations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_11863 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Excess Mean Squared Error of Empirical Bayes Estimators Ju, Yue Wahlberg, Bo Hjalmarsson, Håkan Statistics Theory Systems and Control Empirical Bayes estimators are based on minimizing the average risk with the hyper-parameters in the weighting function being estimated from observed data. The performance of an empirical Bayes estimator is typically evaluated by its mean squared error (MSE). However, the explicit expression for its MSE is generally unavailable for finite sample sizes. To address this issue, we define a high-order analytical criterion: the excess MSE. It quantifies the performance difference between the maximum likelihood and empirical Bayes estimators. An explicit expression for the excess MSE of an empirical Bayes estimator employing a general data-dependent hyper-parameter estimator is derived. As specific instances, we provide excess MSE expressions for kernel-based regularized estimators using the scaled empirical Bayes, Stein unbiased risk estimation, and generalized cross-validation hyper-parameter estimators. Moreover, we propose a modification to the excess MSE expressions for regularized estimators for moderate sample sizes and show its improvement on accuracy in numerical simulations. |
| title | Excess Mean Squared Error of Empirical Bayes Estimators |
| topic | Statistics Theory Systems and Control |
| url | https://arxiv.org/abs/2503.11863 |