On Evaluating the Efficiency of Source Code Generated by LLMs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913307130068992 |
|---|---|
| author | Niu, Changan Zhang, Ting Li, Chuanyi Luo, Bin Ng, Vincent |
| author_facet | Niu, Changan Zhang, Ting Li, Chuanyi Luo, Bin Ng, Vincent |
| contents | Recent years have seen the remarkable capabilities of large language models (LLMs) for code generation. Different from existing work that evaluate the correctness of the code generated by LLMs, we propose to further evaluate its efficiency. More efficient code can lead to higher performance and execution efficiency of programs and software completed by LLM-assisted programming. First, we evaluate the efficiency of the code generated by LLMs on two benchmarks, HumanEval and MBPP. Then, we choose a set of programming problems from the online judge platform LeetCode to conduct a more difficult evaluation. Finally, we explore several prompts that would enable LLMs to generate more efficient code. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_06041 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On Evaluating the Efficiency of Source Code Generated by LLMs Niu, Changan Zhang, Ting Li, Chuanyi Luo, Bin Ng, Vincent Software Engineering Recent years have seen the remarkable capabilities of large language models (LLMs) for code generation. Different from existing work that evaluate the correctness of the code generated by LLMs, we propose to further evaluate its efficiency. More efficient code can lead to higher performance and execution efficiency of programs and software completed by LLM-assisted programming. First, we evaluate the efficiency of the code generated by LLMs on two benchmarks, HumanEval and MBPP. Then, we choose a set of programming problems from the online judge platform LeetCode to conduct a more difficult evaluation. Finally, we explore several prompts that would enable LLMs to generate more efficient code. |
| title | On Evaluating the Efficiency of Source Code Generated by LLMs |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2404.06041 |