Optimal Labeler Assignment and Sampling for Active Learning in the Presence of Imperfect Labels
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909962635051008 |
|---|---|
| author | Ahadi, Pouya Winograd, Blair Zaug, Camille Arora, Karunesh Wang, Lijun Paynabar, Kamran |
| author_facet | Ahadi, Pouya Winograd, Blair Zaug, Camille Arora, Karunesh Wang, Lijun Paynabar, Kamran |
| contents | Active Learning (AL) has garnered significant interest across various application domains where labeling training data is costly. AL provides a framework that helps practitioners query informative samples for annotation by oracles (labelers). However, these labels often contain noise due to varying levels of labeler accuracy. Additionally, uncertain samples are more prone to receiving incorrect labels because of their complexity. Learning from imperfectly labeled data leads to an inaccurate classifier. We propose a novel AL framework to construct a robust classification model by minimizing noise levels. Our approach includes an assignment model that optimally assigns query points to labelers, aiming to minimize the maximum possible noise within each cycle. Additionally, we introduce a new sampling method to identify the best query points, reducing the impact of label noise on classifier performance. Our experiments demonstrate that our approach significantly improves classification performance compared to several benchmark methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_12870 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Optimal Labeler Assignment and Sampling for Active Learning in the Presence of Imperfect Labels Ahadi, Pouya Winograd, Blair Zaug, Camille Arora, Karunesh Wang, Lijun Paynabar, Kamran Machine Learning Artificial Intelligence Optimization and Control Active Learning (AL) has garnered significant interest across various application domains where labeling training data is costly. AL provides a framework that helps practitioners query informative samples for annotation by oracles (labelers). However, these labels often contain noise due to varying levels of labeler accuracy. Additionally, uncertain samples are more prone to receiving incorrect labels because of their complexity. Learning from imperfectly labeled data leads to an inaccurate classifier. We propose a novel AL framework to construct a robust classification model by minimizing noise levels. Our approach includes an assignment model that optimally assigns query points to labelers, aiming to minimize the maximum possible noise within each cycle. Additionally, we introduce a new sampling method to identify the best query points, reducing the impact of label noise on classifier performance. Our experiments demonstrate that our approach significantly improves classification performance compared to several benchmark methods. |
| title | Optimal Labeler Assignment and Sampling for Active Learning in the Presence of Imperfect Labels |
| topic | Machine Learning Artificial Intelligence Optimization and Control |
| url | https://arxiv.org/abs/2512.12870 |