Feedback-Enhanced Online Multiple Testing with Applications to Conformal Selection
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908955992653824 |
|---|---|
| author | Lu, Lin Huo, Yuyang Ren, Haojie Wang, Zhaojun Zou, Changliang |
| author_facet | Lu, Lin Huo, Yuyang Ren, Haojie Wang, Zhaojun Zou, Changliang |
| contents | We study online multiple testing with feedback, where decisions are made sequentially and the true state of the hypothesis is revealed after the decision has been made, either instantly or with a delay. We propose GAIF, a feedback-enhanced generalized alpha-investing framework that dynamically adjusts thresholds using revealed outcomes, ensuring finite-sample false discovery rate (FDR)/marginal FDR control. Extending GAIF to online conformal testing, we construct independent conformal $p$-values and introduce a feedback-driven model selection criterion to identify the best model/score, thereby improving statistical power. We demonstrate the effectiveness of our methods through numerical simulations and real-data applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_03297 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Feedback-Enhanced Online Multiple Testing with Applications to Conformal Selection Lu, Lin Huo, Yuyang Ren, Haojie Wang, Zhaojun Zou, Changliang Methodology Machine Learning We study online multiple testing with feedback, where decisions are made sequentially and the true state of the hypothesis is revealed after the decision has been made, either instantly or with a delay. We propose GAIF, a feedback-enhanced generalized alpha-investing framework that dynamically adjusts thresholds using revealed outcomes, ensuring finite-sample false discovery rate (FDR)/marginal FDR control. Extending GAIF to online conformal testing, we construct independent conformal $p$-values and introduce a feedback-driven model selection criterion to identify the best model/score, thereby improving statistical power. We demonstrate the effectiveness of our methods through numerical simulations and real-data applications. |
| title | Feedback-Enhanced Online Multiple Testing with Applications to Conformal Selection |
| topic | Methodology Machine Learning |
| url | https://arxiv.org/abs/2509.03297 |