Label Noise Robustness of Conformal Prediction
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2022
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916495348465664 |
|---|---|
| author | Einbinder, Bat-Sheva Feldman, Shai Bates, Stephen Angelopoulos, Anastasios N. Gendler, Asaf Romano, Yaniv |
| author_facet | Einbinder, Bat-Sheva Feldman, Shai Bates, Stephen Angelopoulos, Anastasios N. Gendler, Asaf Romano, Yaniv |
| contents | We study the robustness of conformal prediction, a powerful tool for uncertainty quantification, to label noise. Our analysis tackles both regression and classification problems, characterizing when and how it is possible to construct uncertainty sets that correctly cover the unobserved noiseless ground truth labels. We further extend our theory and formulate the requirements for correctly controlling a general loss function, such as the false negative proportion, with noisy labels. Our theory and experiments suggest that conformal prediction and risk-controlling techniques with noisy labels attain conservative risk over the clean ground truth labels whenever the noise is dispersive and increases variability. In other adversarial cases, we can also correct for noise of bounded size in the conformal prediction algorithm in order to ensure achieving the correct risk of the ground truth labels without score or data regularity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2209_14295 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Label Noise Robustness of Conformal Prediction Einbinder, Bat-Sheva Feldman, Shai Bates, Stephen Angelopoulos, Anastasios N. Gendler, Asaf Romano, Yaniv Machine Learning Artificial Intelligence Statistics Theory Methodology We study the robustness of conformal prediction, a powerful tool for uncertainty quantification, to label noise. Our analysis tackles both regression and classification problems, characterizing when and how it is possible to construct uncertainty sets that correctly cover the unobserved noiseless ground truth labels. We further extend our theory and formulate the requirements for correctly controlling a general loss function, such as the false negative proportion, with noisy labels. Our theory and experiments suggest that conformal prediction and risk-controlling techniques with noisy labels attain conservative risk over the clean ground truth labels whenever the noise is dispersive and increases variability. In other adversarial cases, we can also correct for noise of bounded size in the conformal prediction algorithm in order to ensure achieving the correct risk of the ground truth labels without score or data regularity. |
| title | Label Noise Robustness of Conformal Prediction |
| topic | Machine Learning Artificial Intelligence Statistics Theory Methodology |
| url | https://arxiv.org/abs/2209.14295 |