Towards a Science of Collective AI: LLM-based Multi-Agent Systems Need a Transition from Blind Trial-and-Error to Rigorous Science
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911422606213120 |
|---|---|
| author | Fan, Jingru Liu, Dewen Dang, Yufan Li, Huatao Wang, Yuheng Liu, Wei Duan, Feiyu Ding, Xuanwen Yao, Shu Wu, Lin Shi, Ruijie Leung, Wai-Shing Cheng, Yuan Wei, Zhongyu Yang, Cheng Qian, Chen Liu, Zhiyuan Sun, Maosong |
| author_facet | Fan, Jingru Liu, Dewen Dang, Yufan Li, Huatao Wang, Yuheng Liu, Wei Duan, Feiyu Ding, Xuanwen Yao, Shu Wu, Lin Shi, Ruijie Leung, Wai-Shing Cheng, Yuan Wei, Zhongyu Yang, Cheng Qian, Chen Liu, Zhiyuan Sun, Maosong |
| contents | Recent advancements in Large Language Models (LLMs) have greatly extended the capabilities of Multi-Agent Systems (MAS), demonstrating significant effectiveness across a wide range of complex and open-ended domains. However, despite this rapid progress, the field still relies heavily on empirical trial-and-error. It lacks a unified and principled scientific framework necessary for systematic optimization and improvement. This bottleneck stems from the ambiguity of attribution: first, the absence of a structured taxonomy of factors leaves researchers restricted to unguided adjustments; second, the lack of a unified metric fails to distinguish genuine collaboration gain from mere resource accumulation. In this paper, we advocate for a transition to design science through an integrated framework. We advocate to establish the collaboration gain metric ($Γ$) as the scientific standard to isolate intrinsic gains from increased budgets. Leveraging $Γ$, we propose a factor attribution paradigm to systematically identify collaboration-driving factors. To support this, we construct a systematic MAS factor library, structuring the design space into control-level presets and information-level dynamics. Ultimately, this framework facilitates the transition from blind experimentation to rigorous science, paving the way towards a true science of Collective AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_05289 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Towards a Science of Collective AI: LLM-based Multi-Agent Systems Need a Transition from Blind Trial-and-Error to Rigorous Science Fan, Jingru Liu, Dewen Dang, Yufan Li, Huatao Wang, Yuheng Liu, Wei Duan, Feiyu Ding, Xuanwen Yao, Shu Wu, Lin Shi, Ruijie Leung, Wai-Shing Cheng, Yuan Wei, Zhongyu Yang, Cheng Qian, Chen Liu, Zhiyuan Sun, Maosong Computation and Language Artificial Intelligence Multiagent Systems Recent advancements in Large Language Models (LLMs) have greatly extended the capabilities of Multi-Agent Systems (MAS), demonstrating significant effectiveness across a wide range of complex and open-ended domains. However, despite this rapid progress, the field still relies heavily on empirical trial-and-error. It lacks a unified and principled scientific framework necessary for systematic optimization and improvement. This bottleneck stems from the ambiguity of attribution: first, the absence of a structured taxonomy of factors leaves researchers restricted to unguided adjustments; second, the lack of a unified metric fails to distinguish genuine collaboration gain from mere resource accumulation. In this paper, we advocate for a transition to design science through an integrated framework. We advocate to establish the collaboration gain metric ($Γ$) as the scientific standard to isolate intrinsic gains from increased budgets. Leveraging $Γ$, we propose a factor attribution paradigm to systematically identify collaboration-driving factors. To support this, we construct a systematic MAS factor library, structuring the design space into control-level presets and information-level dynamics. Ultimately, this framework facilitates the transition from blind experimentation to rigorous science, paving the way towards a true science of Collective AI. |
| title | Towards a Science of Collective AI: LLM-based Multi-Agent Systems Need a Transition from Blind Trial-and-Error to Rigorous Science |
| topic | Computation and Language Artificial Intelligence Multiagent Systems |
| url | https://arxiv.org/abs/2602.05289 |