Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2506.13824 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918060369117184 |
|---|---|
| author | Huang, Jinyang Feng, Xiachong Chen, Qiguang Zhao, Hanjie Cheng, Zihui Bai, Jiesong Zhou, Jingxuan Li, Min Qin, Libo |
| author_facet | Huang, Jinyang Feng, Xiachong Chen, Qiguang Zhao, Hanjie Cheng, Zihui Bai, Jiesong Zhou, Jingxuan Li, Min Qin, Libo |
| contents | Code debugging is a crucial task in software engineering, which attracts increasing attention. While remarkable success has been made in the era of large language models (LLMs), current research still focuses on the simple no-library or single-library setting, ignoring the complex multi-library scenario in real-world applications. To address this limitation, we make the first attempt to introduce MLDebugging (Multi-Library Debugging), a comprehensive benchmark designed to assess debugging challenges within multi-library Python code. Specifically, MLDebugging encompasses 126 distinct Python libraries, covering a wide range of multi-library code issues, categorized into seven distinct types. Furthermore, we conduct a thorough evaluation of MLDebugging using both mainstream open-source and closed-source LLMs and highlight that current LLMs still struggle to correctly perform code debugging across multi-library scenarios. We hope this work can uncover the potential of LLMs in multi-library debugging scenario and offer insights for future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_13824 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MLDebugging: Towards Benchmarking Code Debugging Across Multi-Library Scenarios Huang, Jinyang Feng, Xiachong Chen, Qiguang Zhao, Hanjie Cheng, Zihui Bai, Jiesong Zhou, Jingxuan Li, Min Qin, Libo Software Engineering Artificial Intelligence Code debugging is a crucial task in software engineering, which attracts increasing attention. While remarkable success has been made in the era of large language models (LLMs), current research still focuses on the simple no-library or single-library setting, ignoring the complex multi-library scenario in real-world applications. To address this limitation, we make the first attempt to introduce MLDebugging (Multi-Library Debugging), a comprehensive benchmark designed to assess debugging challenges within multi-library Python code. Specifically, MLDebugging encompasses 126 distinct Python libraries, covering a wide range of multi-library code issues, categorized into seven distinct types. Furthermore, we conduct a thorough evaluation of MLDebugging using both mainstream open-source and closed-source LLMs and highlight that current LLMs still struggle to correctly perform code debugging across multi-library scenarios. We hope this work can uncover the potential of LLMs in multi-library debugging scenario and offer insights for future research. |
| title | MLDebugging: Towards Benchmarking Code Debugging Across Multi-Library Scenarios |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2506.13824 |