Conformal Prediction of Classifiers with Many Classes based on Noisy Labels
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908487917764608 |
|---|---|
| author | Penso, Coby Goldberger, Jacob Fetaya, Ethan |
| author_facet | Penso, Coby Goldberger, Jacob Fetaya, Ethan |
| contents | Conformal Prediction (CP) controls the prediction uncertainty of classification systems by producing a small prediction set, ensuring a predetermined probability that the true class lies within this set. This is commonly done by defining a score, based on the model predictions, and setting a threshold on this score using a validation set. In this study, we address the problem of CP calibration when we only have access to a calibration set with noisy labels. We show how we can estimate the noise-free conformal threshold based on the noisy labeled data. We derive a finite sample coverage guarantee for uniform noise that remains effective even in tasks with a large number of classes. We dub our approach Noise-Aware Conformal Prediction (NACP). We illustrate the performance of the proposed results on several standard image classification datasets with a large number of classes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_12749 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Conformal Prediction of Classifiers with Many Classes based on Noisy Labels Penso, Coby Goldberger, Jacob Fetaya, Ethan Machine Learning Artificial Intelligence Conformal Prediction (CP) controls the prediction uncertainty of classification systems by producing a small prediction set, ensuring a predetermined probability that the true class lies within this set. This is commonly done by defining a score, based on the model predictions, and setting a threshold on this score using a validation set. In this study, we address the problem of CP calibration when we only have access to a calibration set with noisy labels. We show how we can estimate the noise-free conformal threshold based on the noisy labeled data. We derive a finite sample coverage guarantee for uniform noise that remains effective even in tasks with a large number of classes. We dub our approach Noise-Aware Conformal Prediction (NACP). We illustrate the performance of the proposed results on several standard image classification datasets with a large number of classes. |
| title | Conformal Prediction of Classifiers with Many Classes based on Noisy Labels |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2501.12749 |