Max-Rank: Efficient Multiple Testing for Conformal Prediction
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929762362982400 |
|---|---|
| author | Timans, Alexander Straehle, Christoph-Nikolas Sakmann, Kaspar Naesseth, Christian A. Nalisnick, Eric |
| author_facet | Timans, Alexander Straehle, Christoph-Nikolas Sakmann, Kaspar Naesseth, Christian A. Nalisnick, Eric |
| contents | Multiple hypothesis testing (MHT) frequently arises in scientific inquiries, and concurrent testing of multiple hypotheses inflates the risk of Type-I errors or false positives, rendering MHT corrections essential. This paper addresses MHT in the context of conformal prediction, a flexible framework for predictive uncertainty quantification. Some conformal applications give rise to simultaneous testing, and positive dependencies among tests typically exist. We introduce $\texttt{max-rank}$, a novel correction that exploits these dependencies whilst efficiently controlling the family-wise error rate. Inspired by existing permutation-based corrections, $\texttt{max-rank}$ leverages rank order information to improve performance and integrates readily with any conformal procedure. We establish its theoretical and empirical advantages over the common Bonferroni correction and its compatibility with conformal prediction, highlighting the potential to strengthen predictive uncertainty estimates. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_10900 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Max-Rank: Efficient Multiple Testing for Conformal Prediction Timans, Alexander Straehle, Christoph-Nikolas Sakmann, Kaspar Naesseth, Christian A. Nalisnick, Eric Methodology Statistics Theory Machine Learning Multiple hypothesis testing (MHT) frequently arises in scientific inquiries, and concurrent testing of multiple hypotheses inflates the risk of Type-I errors or false positives, rendering MHT corrections essential. This paper addresses MHT in the context of conformal prediction, a flexible framework for predictive uncertainty quantification. Some conformal applications give rise to simultaneous testing, and positive dependencies among tests typically exist. We introduce $\texttt{max-rank}$, a novel correction that exploits these dependencies whilst efficiently controlling the family-wise error rate. Inspired by existing permutation-based corrections, $\texttt{max-rank}$ leverages rank order information to improve performance and integrates readily with any conformal procedure. We establish its theoretical and empirical advantages over the common Bonferroni correction and its compatibility with conformal prediction, highlighting the potential to strengthen predictive uncertainty estimates. |
| title | Max-Rank: Efficient Multiple Testing for Conformal Prediction |
| topic | Methodology Statistics Theory Machine Learning |
| url | https://arxiv.org/abs/2311.10900 |