Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2408.13690 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929472910917632 |
|---|---|
| author | Rahmati, Amir Hossein Fan, Mingzhou Zhou, Ruida Urban, Nathan M. Yoon, Byung-Jun Qian, Xiaoning |
| author_facet | Rahmati, Amir Hossein Fan, Mingzhou Zhou, Ruida Urban, Nathan M. Yoon, Byung-Jun Qian, Xiaoning |
| contents | Instead of randomly acquiring training data points, Uncertainty-based Active Learning (UAL) operates by querying the label(s) of pivotal samples from an unlabeled pool selected based on the prediction uncertainty, thereby aiming at minimizing the labeling cost for model training. The efficacy of UAL critically depends on the model capacity as well as the adopted uncertainty-based acquisition function. Within the context of this study, our analytical focus is directed toward comprehending how the capacity of the machine learning model may affect UAL efficacy. Through theoretical analysis, comprehensive simulations, and empirical studies, we conclusively demonstrate that UAL can lead to worse performance in comparison with random sampling when the machine learning model class has low capacity and is unable to cover the underlying ground truth. In such situations, adopting acquisition functions that directly target estimating the prediction performance may be beneficial for improving the performance of UAL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_13690 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Understanding Uncertainty-based Active Learning Under Model Mismatch Rahmati, Amir Hossein Fan, Mingzhou Zhou, Ruida Urban, Nathan M. Yoon, Byung-Jun Qian, Xiaoning Machine Learning Instead of randomly acquiring training data points, Uncertainty-based Active Learning (UAL) operates by querying the label(s) of pivotal samples from an unlabeled pool selected based on the prediction uncertainty, thereby aiming at minimizing the labeling cost for model training. The efficacy of UAL critically depends on the model capacity as well as the adopted uncertainty-based acquisition function. Within the context of this study, our analytical focus is directed toward comprehending how the capacity of the machine learning model may affect UAL efficacy. Through theoretical analysis, comprehensive simulations, and empirical studies, we conclusively demonstrate that UAL can lead to worse performance in comparison with random sampling when the machine learning model class has low capacity and is unable to cover the underlying ground truth. In such situations, adopting acquisition functions that directly target estimating the prediction performance may be beneficial for improving the performance of UAL. |
| title | Understanding Uncertainty-based Active Learning Under Model Mismatch |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2408.13690 |