MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913717732507648 |
|---|---|
| author | Zhu, Kunlun Du, Hongyi Hong, Zhaochen Yang, Xiaocheng Guo, Shuyi Wang, Zhe Wang, Zhenhailong Qian, Cheng Tang, Xiangru Ji, Heng You, Jiaxuan |
| author_facet | Zhu, Kunlun Du, Hongyi Hong, Zhaochen Yang, Xiaocheng Guo, Shuyi Wang, Zhe Wang, Zhenhailong Qian, Cheng Tang, Xiangru Ji, Heng You, Jiaxuan |
| contents | Large Language Models (LLMs) have shown remarkable capabilities as autonomous agents, yet existing benchmarks either focus on single-agent tasks or are confined to narrow domains, failing to capture the dynamics of multi-agent coordination and competition. In this paper, we introduce MultiAgentBench, a comprehensive benchmark designed to evaluate LLM-based multi-agent systems across diverse, interactive scenarios. Our framework measures not only task completion but also the quality of collaboration and competition using novel, milestone-based key performance indicators. Moreover, we evaluate various coordination protocols (including star, chain, tree, and graph topologies) and innovative strategies such as group discussion and cognitive planning. Notably, gpt-4o-mini reaches the average highest task score, graph structure performs the best among coordination protocols in the research scenario, and cognitive planning improves milestone achievement rates by 3%. Code and datasets are public available at https://github.com/MultiagentBench/MARBLE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_01935 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents Zhu, Kunlun Du, Hongyi Hong, Zhaochen Yang, Xiaocheng Guo, Shuyi Wang, Zhe Wang, Zhenhailong Qian, Cheng Tang, Xiangru Ji, Heng You, Jiaxuan Multiagent Systems Artificial Intelligence Computation and Language Computers and Society Large Language Models (LLMs) have shown remarkable capabilities as autonomous agents, yet existing benchmarks either focus on single-agent tasks or are confined to narrow domains, failing to capture the dynamics of multi-agent coordination and competition. In this paper, we introduce MultiAgentBench, a comprehensive benchmark designed to evaluate LLM-based multi-agent systems across diverse, interactive scenarios. Our framework measures not only task completion but also the quality of collaboration and competition using novel, milestone-based key performance indicators. Moreover, we evaluate various coordination protocols (including star, chain, tree, and graph topologies) and innovative strategies such as group discussion and cognitive planning. Notably, gpt-4o-mini reaches the average highest task score, graph structure performs the best among coordination protocols in the research scenario, and cognitive planning improves milestone achievement rates by 3%. Code and datasets are public available at https://github.com/MultiagentBench/MARBLE. |
| title | MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents |
| topic | Multiagent Systems Artificial Intelligence Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2503.01935 |