Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915599972564992 |
|---|---|
| author | Zhao, Qianhui Zhang, Li Liu, Fang Cheng, Junhang Wu, Chengru Ai, Junchen Meng, Qiaoyuanhe Zhang, Lichen Lian, Xiaoli Song, Shubin Guo, Yuanping |
| author_facet | Zhao, Qianhui Zhang, Li Liu, Fang Cheng, Junhang Wu, Chengru Ai, Junchen Meng, Qiaoyuanhe Zhang, Lichen Lian, Xiaoli Song, Shubin Guo, Yuanping |
| contents | In recent years, Large Language Models (LLMs) have achieved remarkable progress in automated code generation. In real-world software engineering, the growing demand for rapid iteration and continuous delivery underscores the importance of project-level code generation, where LLMs are expected to generate complete software projects directly from complex user requirements. Although existing studies have made initial explorations, they still face key limitations, including unrealistic datasets and unreliable evaluation metrics that fail to reflect real-world complexity, the semantic gap between human-written requirements and machine-interpretable structures, and difficulties in managing hierarchical dependencies and maintaining quality throughout the generation process. To address these limitations, we first introduce CodeProjectEval, a project-level code generation dataset built from 18 real-world repositories with 12.7 files and 2,388.6 lines of code per task on average, supplemented with documentation and executable test cases for automatic evaluation. We further propose ProjectGen, a multi-agent framework that decomposes projects into architecture design, skeleton generation, and code filling stages with iterative refinement and memory-based context management. Within this framework, we introduce the Semantic Software Architecture Tree (SSAT), a structured and semantically rich representation that effectively bridges user requirements and source code implementation. Experiments show that ProjectGen achieves state-of-the-art performance, passing 52/124 test cases on the small-scale project-level code generation dataset DevBench, a 57% improvement over the baseline approaches, and 310 test cases on CodeProjectEval, representing an improvement of roughly tenfold compared to the baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_03404 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling Zhao, Qianhui Zhang, Li Liu, Fang Cheng, Junhang Wu, Chengru Ai, Junchen Meng, Qiaoyuanhe Zhang, Lichen Lian, Xiaoli Song, Shubin Guo, Yuanping Software Engineering In recent years, Large Language Models (LLMs) have achieved remarkable progress in automated code generation. In real-world software engineering, the growing demand for rapid iteration and continuous delivery underscores the importance of project-level code generation, where LLMs are expected to generate complete software projects directly from complex user requirements. Although existing studies have made initial explorations, they still face key limitations, including unrealistic datasets and unreliable evaluation metrics that fail to reflect real-world complexity, the semantic gap between human-written requirements and machine-interpretable structures, and difficulties in managing hierarchical dependencies and maintaining quality throughout the generation process. To address these limitations, we first introduce CodeProjectEval, a project-level code generation dataset built from 18 real-world repositories with 12.7 files and 2,388.6 lines of code per task on average, supplemented with documentation and executable test cases for automatic evaluation. We further propose ProjectGen, a multi-agent framework that decomposes projects into architecture design, skeleton generation, and code filling stages with iterative refinement and memory-based context management. Within this framework, we introduce the Semantic Software Architecture Tree (SSAT), a structured and semantically rich representation that effectively bridges user requirements and source code implementation. Experiments show that ProjectGen achieves state-of-the-art performance, passing 52/124 test cases on the small-scale project-level code generation dataset DevBench, a 57% improvement over the baseline approaches, and 310 test cases on CodeProjectEval, representing an improvement of roughly tenfold compared to the baselines. |
| title | Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2511.03404 |