Bayesian Transfer Learning for Enhanced Estimation and Inference
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910726597115904 |
|---|---|
| author | Lai, Daoyuan Padilla, Oscar Hernan Madrid Gu, Tian |
| author_facet | Lai, Daoyuan Padilla, Oscar Hernan Madrid Gu, Tian |
| contents | Transfer learning enhances model performance in a target population with limited samples by leveraging knowledge from related studies. While many works focus on improving predictive performance, challenges of statistical inference persist. Bayesian approaches naturally offer uncertainty quantification for parameter estimates, yet existing Bayesian transfer learning methods are typically limited to single-source scenarios or require individual-level data. We introduce TRansfer leArning via guideD horseshoE prioR (TRADER), a novel approach enabling multi-source transfer through pre-trained models in high-dimensional linear regression. TRADER shrinks target parameters towards a weighted average of source estimates, accommodating sources with different scales. Theoretical investigation shows that TRADER achieves faster posterior contraction rates than standard continuous shrinkage priors when sources align well with the target while preventing negative transfer from heterogeneous sources. The analysis of finite-sample marginal posterior behavior reveals that TRADER achieves desired frequentist coverage probabilities, even for coefficients with moderate signal strength--a scenario where standard continuous shrinkage priors struggle. Extensive numerical studies and a real-data application estimating the association between blood glucose and insulin use in the Hispanic diabetic population demonstrate that TRADER improves estimation and inference accuracy over continuous shrinkage priors using target data alone, while outperforming a state-of-the-art transfer learning method that requires individual-level data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_02986 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Bayesian Transfer Learning for Enhanced Estimation and Inference Lai, Daoyuan Padilla, Oscar Hernan Madrid Gu, Tian Methodology Transfer learning enhances model performance in a target population with limited samples by leveraging knowledge from related studies. While many works focus on improving predictive performance, challenges of statistical inference persist. Bayesian approaches naturally offer uncertainty quantification for parameter estimates, yet existing Bayesian transfer learning methods are typically limited to single-source scenarios or require individual-level data. We introduce TRansfer leArning via guideD horseshoE prioR (TRADER), a novel approach enabling multi-source transfer through pre-trained models in high-dimensional linear regression. TRADER shrinks target parameters towards a weighted average of source estimates, accommodating sources with different scales. Theoretical investigation shows that TRADER achieves faster posterior contraction rates than standard continuous shrinkage priors when sources align well with the target while preventing negative transfer from heterogeneous sources. The analysis of finite-sample marginal posterior behavior reveals that TRADER achieves desired frequentist coverage probabilities, even for coefficients with moderate signal strength--a scenario where standard continuous shrinkage priors struggle. Extensive numerical studies and a real-data application estimating the association between blood glucose and insulin use in the Hispanic diabetic population demonstrate that TRADER improves estimation and inference accuracy over continuous shrinkage priors using target data alone, while outperforming a state-of-the-art transfer learning method that requires individual-level data. |
| title | Bayesian Transfer Learning for Enhanced Estimation and Inference |
| topic | Methodology |
| url | https://arxiv.org/abs/2412.02986 |