Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2509.14030 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918143055626240 |
|---|---|
| author | Qin, Maosheng Zhu, Renyu Xia, Mingxuan Chen, Chenkai Zhu, Zhen Lin, Minmin Zhao, Junbo Xu, Lu Fan, Changjie Wu, Runze Wang, Haobo |
| author_facet | Qin, Maosheng Zhu, Renyu Xia, Mingxuan Chen, Chenkai Zhu, Zhen Lin, Minmin Zhao, Junbo Xu, Lu Fan, Changjie Wu, Runze Wang, Haobo |
| contents | High-quality annotated data is a cornerstone of modern Natural Language Processing (NLP). While recent methods begin to leverage diverse annotation sources-including Large Language Models (LLMs), Small Language Models (SLMs), and human experts-they often focus narrowly on the labeling step itself. A critical gap remains in the holistic process control required to manage these sources dynamically, addressing complex scheduling and quality-cost trade-offs in a unified manner. Inspired by real-world crowdsourcing companies, we introduce CrowdAgent, a multi-agent system that provides end-to-end process control by integrating task assignment, data annotation, and quality/cost management. It implements a novel methodology that rationally assigns tasks, enabling LLMs, SLMs, and human experts to advance synergistically in a collaborative annotation workflow. We demonstrate the effectiveness of CrowdAgent through extensive experiments on six diverse multimodal classification tasks. The source code and video demo are available at https://github.com/QMMMS/CrowdAgent. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_14030 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CrowdAgent: Multi-Agent Managed Multi-Source Annotation System Qin, Maosheng Zhu, Renyu Xia, Mingxuan Chen, Chenkai Zhu, Zhen Lin, Minmin Zhao, Junbo Xu, Lu Fan, Changjie Wu, Runze Wang, Haobo Artificial Intelligence High-quality annotated data is a cornerstone of modern Natural Language Processing (NLP). While recent methods begin to leverage diverse annotation sources-including Large Language Models (LLMs), Small Language Models (SLMs), and human experts-they often focus narrowly on the labeling step itself. A critical gap remains in the holistic process control required to manage these sources dynamically, addressing complex scheduling and quality-cost trade-offs in a unified manner. Inspired by real-world crowdsourcing companies, we introduce CrowdAgent, a multi-agent system that provides end-to-end process control by integrating task assignment, data annotation, and quality/cost management. It implements a novel methodology that rationally assigns tasks, enabling LLMs, SLMs, and human experts to advance synergistically in a collaborative annotation workflow. We demonstrate the effectiveness of CrowdAgent through extensive experiments on six diverse multimodal classification tasks. The source code and video demo are available at https://github.com/QMMMS/CrowdAgent. |
| title | CrowdAgent: Multi-Agent Managed Multi-Source Annotation System |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2509.14030 |