Testing the Assumptions of Active Learning for Translation Tasks with Few Samples
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910118481756160 |
|---|---|
| author | Flores, Lorenzo Jaime Yu di-Piano, Cesare Spinoso Ernst, Ori Adelani, David Ifeoluwa Cheung, Jackie Chi Kit |
| author_facet | Flores, Lorenzo Jaime Yu di-Piano, Cesare Spinoso Ernst, Ori Adelani, David Ifeoluwa Cheung, Jackie Chi Kit |
| contents | Active learning (AL) is a training paradigm for selecting unlabeled samples for annotation to improve model performance on a test set, which is useful when only a limited number of samples can be annotated. These algorithms often work by optimizing for the informativeness and diversity of the training data to be annotated. Recent work found that AL strategies fail to outperform random sampling on various language generation tasks when using 100-500 samples. To understand AL's poor performance when only using few samples, we investigate whether the core assumptions underlying AL strategies hold. We find that neither the informativeness nor diversity of the training data, which AL strategies optimize for, are correlated with test set performance. Instead, factors like the ordering of the training samples and interactions with pre-training data have a larger impact on performance. This suggests that future AL methods must take these factors into account in order to work with very few samples. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_08977 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Testing the Assumptions of Active Learning for Translation Tasks with Few Samples Flores, Lorenzo Jaime Yu di-Piano, Cesare Spinoso Ernst, Ori Adelani, David Ifeoluwa Cheung, Jackie Chi Kit Computation and Language Active learning (AL) is a training paradigm for selecting unlabeled samples for annotation to improve model performance on a test set, which is useful when only a limited number of samples can be annotated. These algorithms often work by optimizing for the informativeness and diversity of the training data to be annotated. Recent work found that AL strategies fail to outperform random sampling on various language generation tasks when using 100-500 samples. To understand AL's poor performance when only using few samples, we investigate whether the core assumptions underlying AL strategies hold. We find that neither the informativeness nor diversity of the training data, which AL strategies optimize for, are correlated with test set performance. Instead, factors like the ordering of the training samples and interactions with pre-training data have a larger impact on performance. This suggests that future AL methods must take these factors into account in order to work with very few samples. |
| title | Testing the Assumptions of Active Learning for Translation Tasks with Few Samples |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.08977 |