A Moment-Based Generalization to Post-Prediction Inference
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909686449569792 |
|---|---|
| author | Salerno, Stephen Hoffman, Kentaro Afiaz, Awan Neufeld, Anna McCormick, Tyler H. Leek, Jeffrey T. |
| author_facet | Salerno, Stephen Hoffman, Kentaro Afiaz, Awan Neufeld, Anna McCormick, Tyler H. Leek, Jeffrey T. |
| contents | Artificial intelligence (AI) and machine learning (ML) are increasingly used to generate data for downstream analyses, yet naively treating these predictions as true observations can lead to biased results and incorrect inference. Wang et al. (2020) proposed a method, post-prediction inference, which calibrates inference by modeling the relationship between AI/ML-predicted and observed outcomes in a small, gold-standard sample. Since then, several methods have been developed for inference with predicted data. We revisit Wang et al. in light of these recent developments. We reflect on their assumptions and offer a simple extension of their method which relaxes these assumptions. Our extension (1) yields unbiased point estimates under standard conditions and (2) incorporates a simple scaling factor to preserve calibration variability. In extensive simulations, we show that our method maintains nominal Type I error rates, reduces bias, and achieves proper coverage. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_09119 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Moment-Based Generalization to Post-Prediction Inference Salerno, Stephen Hoffman, Kentaro Afiaz, Awan Neufeld, Anna McCormick, Tyler H. Leek, Jeffrey T. Methodology Machine Learning Artificial intelligence (AI) and machine learning (ML) are increasingly used to generate data for downstream analyses, yet naively treating these predictions as true observations can lead to biased results and incorrect inference. Wang et al. (2020) proposed a method, post-prediction inference, which calibrates inference by modeling the relationship between AI/ML-predicted and observed outcomes in a small, gold-standard sample. Since then, several methods have been developed for inference with predicted data. We revisit Wang et al. in light of these recent developments. We reflect on their assumptions and offer a simple extension of their method which relaxes these assumptions. Our extension (1) yields unbiased point estimates under standard conditions and (2) incorporates a simple scaling factor to preserve calibration variability. In extensive simulations, we show that our method maintains nominal Type I error rates, reduces bias, and achieves proper coverage. |
| title | A Moment-Based Generalization to Post-Prediction Inference |
| topic | Methodology Machine Learning |
| url | https://arxiv.org/abs/2507.09119 |