Automated Prompt Engineering for Cost-Effective Code Generation Using Evolutionary Algorithm
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913965379944448 |
|---|---|
| author | Taherkhani, Hamed Sepindband, Melika Pham, Hung Viet Wang, Song Hemmati, Hadi |
| author_facet | Taherkhani, Hamed Sepindband, Melika Pham, Hung Viet Wang, Song Hemmati, Hadi |
| contents | Large Language Models have seen increasing use in various software development tasks, especially in code generation. The most advanced recent methods attempt to incorporate feedback from code execution into prompts to help guide LLMs in generating correct code in an iterative process. While effective, these methods could be costly due to numerous interactions with the LLM and extensive token usage. To address this issue, we propose an alternative approach named Evolutionary Prompt Engineering for Code (EPiC), which leverages a lightweight evolutionary algorithm to refine the original prompts into improved versions that generate high quality code, with minimal interactions with the LLM. Our evaluation against state-of-the-art (SOTA) LLM based code generation agents shows that EPiC not only achieves up to 6% improvement in pass@k but is also 2-10 times more cost-effective than the baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_11198 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Automated Prompt Engineering for Cost-Effective Code Generation Using Evolutionary Algorithm Taherkhani, Hamed Sepindband, Melika Pham, Hung Viet Wang, Song Hemmati, Hadi Software Engineering Artificial Intelligence Neural and Evolutionary Computing Large Language Models have seen increasing use in various software development tasks, especially in code generation. The most advanced recent methods attempt to incorporate feedback from code execution into prompts to help guide LLMs in generating correct code in an iterative process. While effective, these methods could be costly due to numerous interactions with the LLM and extensive token usage. To address this issue, we propose an alternative approach named Evolutionary Prompt Engineering for Code (EPiC), which leverages a lightweight evolutionary algorithm to refine the original prompts into improved versions that generate high quality code, with minimal interactions with the LLM. Our evaluation against state-of-the-art (SOTA) LLM based code generation agents shows that EPiC not only achieves up to 6% improvement in pass@k but is also 2-10 times more cost-effective than the baselines. |
| title | Automated Prompt Engineering for Cost-Effective Code Generation Using Evolutionary Algorithm |
| topic | Software Engineering Artificial Intelligence Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2408.11198 |