Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.05199 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913912582045696 |
|---|---|
| author | Tao, Leitian Chen, Xiang Yu, Tong Mai, Tung Rossi, Ryan Li, Yixuan Mitra, Saayan |
| author_facet | Tao, Leitian Chen, Xiang Yu, Tong Mai, Tung Rossi, Ryan Li, Yixuan Mitra, Saayan |
| contents | Large Language Models (LLMs) have revolutionized code generation but require significant resources and often over-generalize, limiting their task-specific efficiency. Fine-tuning smaller, open-source LLMs provides a cost-effective alternative. However, standard supervised approaches rely only on correct examples, missing valuable insights from failures. We introduce CodeLutra, a framework that leverages both correct and incorrect code attempts. Instead of using only correct solutions, CodeLutra applies iterative preference-based refinement, comparing successful and failed outputs to better approximate desired results. This approach narrows the performance gap with state-of-the-art larger models without requiring massive datasets or auxiliary models. For instance, on a challenging data science coding task, using only 500 samples improved Llama-3-8B's accuracy from 28.2% to 48.6%, approaching GPT-4's level. By learning from both successes and mistakes, CodeLutra provides a scalable and efficient path to high-quality code generation, making smaller open-source models more competitive with leading closed-source alternatives. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_05199 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement Tao, Leitian Chen, Xiang Yu, Tong Mai, Tung Rossi, Ryan Li, Yixuan Mitra, Saayan Computation and Language Large Language Models (LLMs) have revolutionized code generation but require significant resources and often over-generalize, limiting their task-specific efficiency. Fine-tuning smaller, open-source LLMs provides a cost-effective alternative. However, standard supervised approaches rely only on correct examples, missing valuable insights from failures. We introduce CodeLutra, a framework that leverages both correct and incorrect code attempts. Instead of using only correct solutions, CodeLutra applies iterative preference-based refinement, comparing successful and failed outputs to better approximate desired results. This approach narrows the performance gap with state-of-the-art larger models without requiring massive datasets or auxiliary models. For instance, on a challenging data science coding task, using only 500 samples improved Llama-3-8B's accuracy from 28.2% to 48.6%, approaching GPT-4's level. By learning from both successes and mistakes, CodeLutra provides a scalable and efficient path to high-quality code generation, making smaller open-source models more competitive with leading closed-source alternatives. |
| title | CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2411.05199 |