Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.02783 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916997570232320 |
|---|---|
| author | Wu, Jie Li, Haoling Zhang, Xin Liu, Xiao Huang, Yangyu Luo, Jianwen Zhang, Yizhen Li, Zuchao Chu, Ruihang Yang, Yujiu Li, Scarlett |
| author_facet | Wu, Jie Li, Haoling Zhang, Xin Liu, Xiao Huang, Yangyu Luo, Jianwen Zhang, Yizhen Li, Zuchao Chu, Ruihang Yang, Yujiu Li, Scarlett |
| contents | Preference learning extends the performance of Code LLMs beyond traditional supervised fine-tuning by leveraging relative quality comparisons. In existing approaches, a set of n candidate solutions is evaluated based on test case success rates, with the candidate demonstrating a higher pass rate being labeled as positive and its counterpart with a lower pass rate as negative. However, because this approach aligns entire failing code blocks rather than pinpointing specific errors, it lacks the granularity necessary to capture meaningful error-correction relationships. As a result, the model is unable to learn more informative error-correction patterns. To address these issues, we propose Target-DPO, a new preference alignment framework that mimics human iterative debugging to refine Code LLMs. Target-DPO explicitly locates error regions and aligns the corresponding tokens via a tailored DPO algorithm. To facilitate it, we introduce the CodeFlow dataset, where samples are iteratively refined until passing tests, with modifications capturing error corrections. Extensive experiments show that a diverse suite of Code LLMs equipped with Target-DPO achieves significant performance gains in code generation and improves on challenging tasks like BigCodeBench. In-depth analysis reveals that Target-DPO yields fewer errors. Code, model and datasets are in: https://github.com/JieWu02/Target-DPO. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_02783 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Teaching Your Models to Understand Code via Focal Preference Alignment Wu, Jie Li, Haoling Zhang, Xin Liu, Xiao Huang, Yangyu Luo, Jianwen Zhang, Yizhen Li, Zuchao Chu, Ruihang Yang, Yujiu Li, Scarlett Computation and Language Artificial Intelligence Machine Learning Preference learning extends the performance of Code LLMs beyond traditional supervised fine-tuning by leveraging relative quality comparisons. In existing approaches, a set of n candidate solutions is evaluated based on test case success rates, with the candidate demonstrating a higher pass rate being labeled as positive and its counterpart with a lower pass rate as negative. However, because this approach aligns entire failing code blocks rather than pinpointing specific errors, it lacks the granularity necessary to capture meaningful error-correction relationships. As a result, the model is unable to learn more informative error-correction patterns. To address these issues, we propose Target-DPO, a new preference alignment framework that mimics human iterative debugging to refine Code LLMs. Target-DPO explicitly locates error regions and aligns the corresponding tokens via a tailored DPO algorithm. To facilitate it, we introduce the CodeFlow dataset, where samples are iteratively refined until passing tests, with modifications capturing error corrections. Extensive experiments show that a diverse suite of Code LLMs equipped with Target-DPO achieves significant performance gains in code generation and improves on challenging tasks like BigCodeBench. In-depth analysis reveals that Target-DPO yields fewer errors. Code, model and datasets are in: https://github.com/JieWu02/Target-DPO. |
| title | Teaching Your Models to Understand Code via Focal Preference Alignment |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2503.02783 |