Self-Evolving Multi-Agent Collaboration Networks for Software Development
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910662639222784 |
|---|---|
| author | Hu, Yue Cai, Yuzhu Du, Yaxin Zhu, Xinyu Liu, Xiangrui Yu, Zijie Hou, Yuchen Tang, Shuo Chen, Siheng |
| author_facet | Hu, Yue Cai, Yuzhu Du, Yaxin Zhu, Xinyu Liu, Xiangrui Yu, Zijie Hou, Yuchen Tang, Shuo Chen, Siheng |
| contents | LLM-driven multi-agent collaboration (MAC) systems have demonstrated impressive capabilities in automatic software development at the function level. However, their heavy reliance on human design limits their adaptability to the diverse demands of real-world software development. To address this limitation, we introduce EvoMAC, a novel self-evolving paradigm for MAC networks. Inspired by traditional neural network training, EvoMAC obtains text-based environmental feedback by verifying the MAC network's output against a target proxy and leverages a novel textual backpropagation to update the network. To extend coding capabilities beyond function-level tasks to more challenging software-level development, we further propose rSDE-Bench, a requirement-oriented software development benchmark, which features complex and diverse software requirements along with automatic evaluation of requirement correctness. Our experiments show that: i) The automatic requirement-aware evaluation in rSDE-Bench closely aligns with human evaluations, validating its reliability as a software-level coding benchmark. ii) EvoMAC outperforms previous SOTA methods on both the software-level rSDE-Bench and the function-level HumanEval benchmarks, reflecting its superior coding capabilities. The benchmark can be downloaded at https://yuzhu-cai.github.io/rSDE-Bench/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_16946 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Self-Evolving Multi-Agent Collaboration Networks for Software Development Hu, Yue Cai, Yuzhu Du, Yaxin Zhu, Xinyu Liu, Xiangrui Yu, Zijie Hou, Yuchen Tang, Shuo Chen, Siheng Software Engineering Artificial Intelligence Multiagent Systems LLM-driven multi-agent collaboration (MAC) systems have demonstrated impressive capabilities in automatic software development at the function level. However, their heavy reliance on human design limits their adaptability to the diverse demands of real-world software development. To address this limitation, we introduce EvoMAC, a novel self-evolving paradigm for MAC networks. Inspired by traditional neural network training, EvoMAC obtains text-based environmental feedback by verifying the MAC network's output against a target proxy and leverages a novel textual backpropagation to update the network. To extend coding capabilities beyond function-level tasks to more challenging software-level development, we further propose rSDE-Bench, a requirement-oriented software development benchmark, which features complex and diverse software requirements along with automatic evaluation of requirement correctness. Our experiments show that: i) The automatic requirement-aware evaluation in rSDE-Bench closely aligns with human evaluations, validating its reliability as a software-level coding benchmark. ii) EvoMAC outperforms previous SOTA methods on both the software-level rSDE-Bench and the function-level HumanEval benchmarks, reflecting its superior coding capabilities. The benchmark can be downloaded at https://yuzhu-cai.github.io/rSDE-Bench/. |
| title | Self-Evolving Multi-Agent Collaboration Networks for Software Development |
| topic | Software Engineering Artificial Intelligence Multiagent Systems |
| url | https://arxiv.org/abs/2410.16946 |