Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912410798915584 |
|---|---|
| author | He, Mengliang Zeng, Jiayi Jiang, Yankai Zhang, Wei Liu, Zeming Shi, Xiaoming Zhou, Aimin |
| author_facet | He, Mengliang Zeng, Jiayi Jiang, Yankai Zhang, Wei Liu, Zeming Shi, Xiaoming Zhou, Aimin |
| contents | While large language models (LLMs) show promise in code generation, existing benchmarks neglect the flowchart-based code generation. To promote further research on flowchart-based code generation, this work presents Flow2Code, a novel benchmark for flowchart-based code generation evaluation. The evaluation dataset spans 15 programming languages and includes 5,622 code segments paired with 16,866 flowcharts of three types: code, UML, and pseudocode. Extensive experiments with 13 multimodal LLMs reveal that current LLMs can not generate code based on flowcharts perfectly. Besides, experiment results show that the supervised fine-tuning technique contributes greatly to the models' performance. We publicly release our code and datasets at https://github.com/hml-github/Flow2Code. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_02073 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability He, Mengliang Zeng, Jiayi Jiang, Yankai Zhang, Wei Liu, Zeming Shi, Xiaoming Zhou, Aimin Software Engineering Artificial Intelligence While large language models (LLMs) show promise in code generation, existing benchmarks neglect the flowchart-based code generation. To promote further research on flowchart-based code generation, this work presents Flow2Code, a novel benchmark for flowchart-based code generation evaluation. The evaluation dataset spans 15 programming languages and includes 5,622 code segments paired with 16,866 flowcharts of three types: code, UML, and pseudocode. Extensive experiments with 13 multimodal LLMs reveal that current LLMs can not generate code based on flowcharts perfectly. Besides, experiment results show that the supervised fine-tuning technique contributes greatly to the models' performance. We publicly release our code and datasets at https://github.com/hml-github/Flow2Code. |
| title | Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2506.02073 |