Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915354093027328 |
|---|---|
| author | Wang, Yanlin Jiang, Tianyue Liu, Mingwei Chen, Jiachi Mao, Mingzhi Liu, Xilin Ma, Yuchi Zheng, Zibin |
| author_facet | Wang, Yanlin Jiang, Tianyue Liu, Mingwei Chen, Jiachi Mao, Mingzhi Liu, Xilin Ma, Yuchi Zheng, Zibin |
| contents | Large language models (LLMs) have brought a paradigm shift to the field of code generation, offering the potential to enhance the software development process. However, previous research mainly focuses on the accuracy of code generation, while coding style differences between LLMs and human developers remain under-explored. In this paper, we empirically analyze the differences in coding style between the code generated by mainstream Code LLMs and the code written by human developers, and summarize coding style inconsistency taxonomy. Specifically, we first summarize the types of coding style inconsistencies by manually analyzing a large number of generation results. We then compare the code generated by Code LLMs with the code written by human programmers in terms of readability, conciseness, and robustness. The results reveal that LLMs and developers have different coding styles. Additionally, we study the possible causes of these inconsistencies and provide some solutions to alleviate the problem. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_00456 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models Wang, Yanlin Jiang, Tianyue Liu, Mingwei Chen, Jiachi Mao, Mingzhi Liu, Xilin Ma, Yuchi Zheng, Zibin Software Engineering Artificial Intelligence Large language models (LLMs) have brought a paradigm shift to the field of code generation, offering the potential to enhance the software development process. However, previous research mainly focuses on the accuracy of code generation, while coding style differences between LLMs and human developers remain under-explored. In this paper, we empirically analyze the differences in coding style between the code generated by mainstream Code LLMs and the code written by human developers, and summarize coding style inconsistency taxonomy. Specifically, we first summarize the types of coding style inconsistencies by manually analyzing a large number of generation results. We then compare the code generated by Code LLMs with the code written by human programmers in terms of readability, conciseness, and robustness. The results reveal that LLMs and developers have different coding styles. Additionally, we study the possible causes of these inconsistencies and provide some solutions to alleviate the problem. |
| title | Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2407.00456 |