Performance Review on LLM for solving leetcode problems
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909519716548608 |
|---|---|
| author | Wang, Lun Shi, Chuanqi Du, Shaoshui Tao, Yiyi Shen, Yixian Zheng, Hang Shen, Yanxin Qiu, Xinyu |
| author_facet | Wang, Lun Shi, Chuanqi Du, Shaoshui Tao, Yiyi Shen, Yixian Zheng, Hang Shen, Yanxin Qiu, Xinyu |
| contents | This paper presents a comprehensive performance evaluation of Large Language Models (LLMs) in solving programming challenges from Leetcode, a widely used platform for algorithm practice and technical interviews. We began by crawling the Leetcode website to collect a diverse set of problems encompassing various difficulty levels and topics. Using this dataset, we generated solutions with multiple LLMs, including GPT-4 and GPT-3.5-turbo (ChatGPT-turbo). The generated solutions were systematically evaluated for correctness and efficiency. We employed the pass@k metric to assess the success rates within a given number of attempts and analyzed the runtime performance of the solutions. Our results highlight the strengths and limitations of current LLMs [10] in code generation and problem-solving tasks, providing insights into their potential applications and areas for improvement in automated programming assistance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_15770 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Performance Review on LLM for solving leetcode problems Wang, Lun Shi, Chuanqi Du, Shaoshui Tao, Yiyi Shen, Yixian Zheng, Hang Shen, Yanxin Qiu, Xinyu Software Engineering Artificial Intelligence This paper presents a comprehensive performance evaluation of Large Language Models (LLMs) in solving programming challenges from Leetcode, a widely used platform for algorithm practice and technical interviews. We began by crawling the Leetcode website to collect a diverse set of problems encompassing various difficulty levels and topics. Using this dataset, we generated solutions with multiple LLMs, including GPT-4 and GPT-3.5-turbo (ChatGPT-turbo). The generated solutions were systematically evaluated for correctness and efficiency. We employed the pass@k metric to assess the success rates within a given number of attempts and analyzed the runtime performance of the solutions. Our results highlight the strengths and limitations of current LLMs [10] in code generation and problem-solving tasks, providing insights into their potential applications and areas for improvement in automated programming assistance. |
| title | Performance Review on LLM for solving leetcode problems |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2502.15770 |