Identifying and Mitigating API Misuse in Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917152761577472 |
|---|---|
| author | Zhuo, Terry Yue He, Junda Sun, Jiamou Xing, Zhenchang Lo, David Grundy, John Du, Xiaoning |
| author_facet | Zhuo, Terry Yue He, Junda Sun, Jiamou Xing, Zhenchang Lo, David Grundy, John Du, Xiaoning |
| contents | API misuse in code generated by large language models (LLMs) presents a serious and growing challenge in software development, as although LLMs demonstrate impressive code generation capabilities, their interactions with complex library APIs are often error-prone and can lead to software failures and vulnerabilities. In this paper, we conduct a large-scale study of API misuse patterns in LLM-generated code by analyzing both method selection and parameter usage across Python and Java, using three representative LLMs: StarCoder-7B, Qwen2.5-Coder-7B, and GitHub Copilot. Based on extensive manual annotation of 3,209 method-level and 3,492 parameter-level misuses, we identify and categorize four recurring misuse types by building on and refining prior API misuse taxonomies. Our evaluation of the three LLMs reveals persistent challenges in API usage, particularly hallucination and intent misalignment. To address these issues, we propose Dr.Fix, an LLM-based automatic repair approach guided by our taxonomy, which improves repair accuracy compared to baseline prompting and existing repair methods, achieving gains of up to 38.4 BLEU and 40% exact match on benchmark datasets. This work offers important insights into the current limitations of LLMs in API usage and points to directions for improving automated misuse repair in code generation systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_22821 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Identifying and Mitigating API Misuse in Large Language Models Zhuo, Terry Yue He, Junda Sun, Jiamou Xing, Zhenchang Lo, David Grundy, John Du, Xiaoning Software Engineering API misuse in code generated by large language models (LLMs) presents a serious and growing challenge in software development, as although LLMs demonstrate impressive code generation capabilities, their interactions with complex library APIs are often error-prone and can lead to software failures and vulnerabilities. In this paper, we conduct a large-scale study of API misuse patterns in LLM-generated code by analyzing both method selection and parameter usage across Python and Java, using three representative LLMs: StarCoder-7B, Qwen2.5-Coder-7B, and GitHub Copilot. Based on extensive manual annotation of 3,209 method-level and 3,492 parameter-level misuses, we identify and categorize four recurring misuse types by building on and refining prior API misuse taxonomies. Our evaluation of the three LLMs reveals persistent challenges in API usage, particularly hallucination and intent misalignment. To address these issues, we propose Dr.Fix, an LLM-based automatic repair approach guided by our taxonomy, which improves repair accuracy compared to baseline prompting and existing repair methods, achieving gains of up to 38.4 BLEU and 40% exact match on benchmark datasets. This work offers important insights into the current limitations of LLMs in API usage and points to directions for improving automated misuse repair in code generation systems. |
| title | Identifying and Mitigating API Misuse in Large Language Models |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2503.22821 |