Alignment with Fill-In-the-Middle for Enhancing Code Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916920752603136 |
|---|---|
| author | Ren, Houxing Lu, Zimu Shi, Weikang Hou, Haotian Yang, Yunqiao Wang, Ke Zhou, Aojun Pan, Junting Zhan, Mingjie Li, Hongsheng |
| author_facet | Ren, Houxing Lu, Zimu Shi, Weikang Hou, Haotian Yang, Yunqiao Wang, Ke Zhou, Aojun Pan, Junting Zhan, Mingjie Li, Hongsheng |
| contents | The code generation capabilities of Large Language Models (LLMs) have advanced applications like tool invocation and problem-solving. However, improving performance in code-related tasks remains challenging due to limited training data that is verifiable with accurate test cases. While Direct Preference Optimization (DPO) has shown promise, existing methods for generating test cases still face limitations. In this paper, we propose a novel approach that splits code snippets into smaller, granular blocks, creating more diverse DPO pairs from the same test cases. Additionally, we introduce the Abstract Syntax Tree (AST) splitting and curriculum training method to enhance the DPO training. Our approach demonstrates significant improvements in code generation tasks, as validated by experiments on benchmark datasets such as HumanEval (+), MBPP (+), APPS, LiveCodeBench, and BigCodeBench. Code and data are available at https://github.com/SenseLLM/StructureCoder. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_19532 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Alignment with Fill-In-the-Middle for Enhancing Code Generation Ren, Houxing Lu, Zimu Shi, Weikang Hou, Haotian Yang, Yunqiao Wang, Ke Zhou, Aojun Pan, Junting Zhan, Mingjie Li, Hongsheng Computation and Language The code generation capabilities of Large Language Models (LLMs) have advanced applications like tool invocation and problem-solving. However, improving performance in code-related tasks remains challenging due to limited training data that is verifiable with accurate test cases. While Direct Preference Optimization (DPO) has shown promise, existing methods for generating test cases still face limitations. In this paper, we propose a novel approach that splits code snippets into smaller, granular blocks, creating more diverse DPO pairs from the same test cases. Additionally, we introduce the Abstract Syntax Tree (AST) splitting and curriculum training method to enhance the DPO training. Our approach demonstrates significant improvements in code generation tasks, as validated by experiments on benchmark datasets such as HumanEval (+), MBPP (+), APPS, LiveCodeBench, and BigCodeBench. Code and data are available at https://github.com/SenseLLM/StructureCoder. |
| title | Alignment with Fill-In-the-Middle for Enhancing Code Generation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2508.19532 |