Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.15773 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915747098263552 |
|---|---|
| author | Qi, Yuanyuan Yang, Xiaohao Lu, Jueqing Guo, Guoxiang Enticott, Joanne Liu, Gang Du, Lan |
| author_facet | Qi, Yuanyuan Yang, Xiaohao Lu, Jueqing Guo, Guoxiang Enticott, Joanne Liu, Gang Du, Lan |
| contents | With the rapid advancement and strong generalization capabilities of large language models (LLMs), they have been increasingly incorporated into the active learning pipelines as annotators to reduce annotation costs. However, considering the annotation quality, labels generated by LLMs often fall short of real-world applicability. To address this, we propose a novel active learning framework, Mixture of LLMs in the Loop Active Learning, replacing human annotators with labels generated through a Mixture-of-LLMs-based annotation model, aimed at enhancing LLM-based annotation robustness by aggregating the strengths of multiple LLMs. To further mitigate the impact of the noisy labels, we introduce annotation discrepancy and negative learning to identify the unreliable annotations and enhance learning effectiveness. Extensive experiments demonstrate that our framework achieves performance comparable to human annotation and consistently outperforms single-LLM baselines and other LLM-ensemble-based approaches. Moreover, our framework is built on lightweight LLMs, enabling it to operate fully on local machines in real-world applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_15773 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Next Generation Active Learning: Mixture of LLMs in the Loop Qi, Yuanyuan Yang, Xiaohao Lu, Jueqing Guo, Guoxiang Enticott, Joanne Liu, Gang Du, Lan Machine Learning With the rapid advancement and strong generalization capabilities of large language models (LLMs), they have been increasingly incorporated into the active learning pipelines as annotators to reduce annotation costs. However, considering the annotation quality, labels generated by LLMs often fall short of real-world applicability. To address this, we propose a novel active learning framework, Mixture of LLMs in the Loop Active Learning, replacing human annotators with labels generated through a Mixture-of-LLMs-based annotation model, aimed at enhancing LLM-based annotation robustness by aggregating the strengths of multiple LLMs. To further mitigate the impact of the noisy labels, we introduce annotation discrepancy and negative learning to identify the unreliable annotations and enhance learning effectiveness. Extensive experiments demonstrate that our framework achieves performance comparable to human annotation and consistently outperforms single-LLM baselines and other LLM-ensemble-based approaches. Moreover, our framework is built on lightweight LLMs, enabling it to operate fully on local machines in real-world applications. |
| title | Next Generation Active Learning: Mixture of LLMs in the Loop |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2601.15773 |