Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.01101 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929592950849536 |
|---|---|
| author | Wen, Ziting Pizarro, Oscar Williams, Stefan |
| author_facet | Wen, Ziting Pizarro, Oscar Williams, Stefan |
| contents | Fine-tuning the pre-trained model with active learning holds promise for reducing annotation costs. However, this combination introduces significant computational costs, particularly with the growing scale of pre-trained models. Recent research has proposed proxy-based active learning, which pre-computes features to reduce computational costs. Yet, this approach often incurs a significant loss in active learning performance, sometimes outweighing the computational cost savings. This paper demonstrates that not all sample selection differences result in performance degradation. Furthermore, we show that suitable training methods can mitigate the decline of active learning performance caused by certain selection discrepancies. Building upon detailed analysis, we propose a novel method, aligned selection via proxy, which improves proxy-based active learning performance by updating pre-computed features and selecting a proper training method. Extensive experiments validate that our method improves the total cost of efficient active learning while maintaining computational efficiency. The code is available at \url{https://github.com/ZiTingW/asvp}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_01101 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Feature Alignment: Rethinking Efficient Active Learning via Proxy in the Context of Pre-trained Models Wen, Ziting Pizarro, Oscar Williams, Stefan Machine Learning Artificial Intelligence Fine-tuning the pre-trained model with active learning holds promise for reducing annotation costs. However, this combination introduces significant computational costs, particularly with the growing scale of pre-trained models. Recent research has proposed proxy-based active learning, which pre-computes features to reduce computational costs. Yet, this approach often incurs a significant loss in active learning performance, sometimes outweighing the computational cost savings. This paper demonstrates that not all sample selection differences result in performance degradation. Furthermore, we show that suitable training methods can mitigate the decline of active learning performance caused by certain selection discrepancies. Building upon detailed analysis, we propose a novel method, aligned selection via proxy, which improves proxy-based active learning performance by updating pre-computed features and selecting a proper training method. Extensive experiments validate that our method improves the total cost of efficient active learning while maintaining computational efficiency. The code is available at \url{https://github.com/ZiTingW/asvp}. |
| title | Feature Alignment: Rethinking Efficient Active Learning via Proxy in the Context of Pre-trained Models |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2403.01101 |