Breaking the Winner's Curse with Bayesian Hybrid Shrinkage
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915607065133056 |
|---|---|
| author | Mudd, Richard Friedberg, Rina Gorbachev, Ilya Nassif, Houssam Zaidi, Abbas |
| author_facet | Mudd, Richard Friedberg, Rina Gorbachev, Ilya Nassif, Houssam Zaidi, Abbas |
| contents | A 'Winner's Curse' arises in large-scale online experimentation platforms when the same experiments are used to both select treatments and evaluate their effects. In these settings, classical difference-in-means estimators of treatment effects are upwardly biased and conventional confidence intervals are rendered invalid. The bias scales with the magnitude of sampling variability and the selection threshold, and inversely with the treatment's true effect size. We propose a new Bayesian approach that incorporates experiment-specific 'local shrinkage' factors that mitigate sensitivity to the choice of prior and improve robustness to assumption violations. We demonstrate how the associated posterior distribution can be estimated without numerical integration techniques, making it a practical choice for at-scale deployment. Through simulation, we evaluate the performance of our approach under various scenarios and find that it performs well even when assumptions about the sampling and selection processes are violated. In an empirical evaluation, our approach demonstrated superior performance over alternative methods, providing more accurate estimates with well-calibrated uncertainty quantification. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_06318 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Breaking the Winner's Curse with Bayesian Hybrid Shrinkage Mudd, Richard Friedberg, Rina Gorbachev, Ilya Nassif, Houssam Zaidi, Abbas Methodology Applications A 'Winner's Curse' arises in large-scale online experimentation platforms when the same experiments are used to both select treatments and evaluate their effects. In these settings, classical difference-in-means estimators of treatment effects are upwardly biased and conventional confidence intervals are rendered invalid. The bias scales with the magnitude of sampling variability and the selection threshold, and inversely with the treatment's true effect size. We propose a new Bayesian approach that incorporates experiment-specific 'local shrinkage' factors that mitigate sensitivity to the choice of prior and improve robustness to assumption violations. We demonstrate how the associated posterior distribution can be estimated without numerical integration techniques, making it a practical choice for at-scale deployment. Through simulation, we evaluate the performance of our approach under various scenarios and find that it performs well even when assumptions about the sampling and selection processes are violated. In an empirical evaluation, our approach demonstrated superior performance over alternative methods, providing more accurate estimates with well-calibrated uncertainty quantification. |
| title | Breaking the Winner's Curse with Bayesian Hybrid Shrinkage |
| topic | Methodology Applications |
| url | https://arxiv.org/abs/2511.06318 |