LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929261783285760 |
|---|---|
| author | Zhang, Jifan Chen, Yifang Canal, Gregory Mussmann, Stephen Das, Arnav M. Bhatt, Gantavya Zhu, Yinglun Bilmes, Jeffrey Du, Simon Shaolei Jamieson, Kevin Nowak, Robert D |
| author_facet | Zhang, Jifan Chen, Yifang Canal, Gregory Mussmann, Stephen Das, Arnav M. Bhatt, Gantavya Zhu, Yinglun Bilmes, Jeffrey Du, Simon Shaolei Jamieson, Kevin Nowak, Robert D |
| contents | Labeled data are critical to modern machine learning applications, but obtaining labels can be expensive. To mitigate this cost, machine learning methods, such as transfer learning, semi-supervised learning and active learning, aim to be label-efficient: achieving high predictive performance from relatively few labeled examples. While obtaining the best label-efficiency in practice often requires combinations of these techniques, existing benchmark and evaluation frameworks do not capture a concerted combination of all such techniques. This paper addresses this deficiency by introducing LabelBench, a new computationally-efficient framework for joint evaluation of multiple label-efficient learning techniques. As an application of LabelBench, we introduce a novel benchmark of state-of-the-art active learning methods in combination with semi-supervised learning for fine-tuning pretrained vision transformers. Our benchmark demonstrates better label-efficiencies than previously reported in active learning. LabelBench's modular codebase is open-sourced for the broader community to contribute label-efficient learning methods and benchmarks. The repository can be found at: https://github.com/EfficientTraining/LabelBench. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_09910 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning Zhang, Jifan Chen, Yifang Canal, Gregory Mussmann, Stephen Das, Arnav M. Bhatt, Gantavya Zhu, Yinglun Bilmes, Jeffrey Du, Simon Shaolei Jamieson, Kevin Nowak, Robert D Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Labeled data are critical to modern machine learning applications, but obtaining labels can be expensive. To mitigate this cost, machine learning methods, such as transfer learning, semi-supervised learning and active learning, aim to be label-efficient: achieving high predictive performance from relatively few labeled examples. While obtaining the best label-efficiency in practice often requires combinations of these techniques, existing benchmark and evaluation frameworks do not capture a concerted combination of all such techniques. This paper addresses this deficiency by introducing LabelBench, a new computationally-efficient framework for joint evaluation of multiple label-efficient learning techniques. As an application of LabelBench, we introduce a novel benchmark of state-of-the-art active learning methods in combination with semi-supervised learning for fine-tuning pretrained vision transformers. Our benchmark demonstrates better label-efficiencies than previously reported in active learning. LabelBench's modular codebase is open-sourced for the broader community to contribute label-efficient learning methods and benchmarks. The repository can be found at: https://github.com/EfficientTraining/LabelBench. |
| title | LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2306.09910 |