LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912337790763008 |
|---|---|
| author | Xia, Yunhui Shen, Wei Wang, Yan Liu, Jason Klein Sun, Huifeng Wu, Siyue Hu, Jian Xu, Xiaolong |
| author_facet | Xia, Yunhui Shen, Wei Wang, Yan Liu, Jason Klein Sun, Huifeng Wu, Siyue Hu, Jian Xu, Xiaolong |
| contents | We introduce LeetCodeDataset, a high-quality benchmark for evaluating and training code-generation models, addressing two key challenges in LLM research: the lack of reasoning-focused coding benchmarks and self-contained training testbeds. By curating LeetCode Python problems with rich metadata, broad coverage, 100+ test cases per problem, and temporal splits (pre/post July 2024), our dataset enables contamination-free evaluation and efficient supervised fine-tuning (SFT). Experiments show reasoning models significantly outperform non-reasoning counterparts, while SFT with only 2.6K model-generated solutions achieves performance comparable to 110K-sample counterparts. The dataset and evaluation framework are available on Hugging Face and Github. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_14655 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMs Xia, Yunhui Shen, Wei Wang, Yan Liu, Jason Klein Sun, Huifeng Wu, Siyue Hu, Jian Xu, Xiaolong Machine Learning Computation and Language Software Engineering We introduce LeetCodeDataset, a high-quality benchmark for evaluating and training code-generation models, addressing two key challenges in LLM research: the lack of reasoning-focused coding benchmarks and self-contained training testbeds. By curating LeetCode Python problems with rich metadata, broad coverage, 100+ test cases per problem, and temporal splits (pre/post July 2024), our dataset enables contamination-free evaluation and efficient supervised fine-tuning (SFT). Experiments show reasoning models significantly outperform non-reasoning counterparts, while SFT with only 2.6K model-generated solutions achieves performance comparable to 110K-sample counterparts. The dataset and evaluation framework are available on Hugging Face and Github. |
| title | LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMs |
| topic | Machine Learning Computation and Language Software Engineering |
| url | https://arxiv.org/abs/2504.14655 |