Case2Code: Scalable Synthetic Data for Code Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913683687342080 |
|---|---|
| author | Shao, Yunfan Li, Linyang Ma, Yichuan Li, Peiji Song, Demin Cheng, Qinyuan Li, Shimin Li, Xiaonan Wang, Pengyu Guo, Qipeng Yan, Hang Qiu, Xipeng Huang, Xuanjing Lin, Dahua |
| author_facet | Shao, Yunfan Li, Linyang Ma, Yichuan Li, Peiji Song, Demin Cheng, Qinyuan Li, Shimin Li, Xiaonan Wang, Pengyu Guo, Qipeng Yan, Hang Qiu, Xipeng Huang, Xuanjing Lin, Dahua |
| contents | Large Language Models (LLMs) have shown outstanding breakthroughs in code generation. Recent work improves code LLMs by training on synthetic data generated by some powerful LLMs, which can be challenging to scale due to the dependence on a teacher model and high generation costs. In this paper, we focus on synthesizing code data at scale and propose a \textbf{Case2Code} task by exploiting the expressiveness and correctness of programs. \textbf{Case2Code} is an inductive inference task that aims to infer underlying code implementations by observing input-output examples or program behaviors, By incorporating LLMs to generate program inputs, and executing the program with these inputs to obtain the program outputs, we can synthesize diverse and high-quality \textbf{Case2Code} data at scale for training and evaluating code LLMs. Experimental results show that case-to-code induction is challenging for current representative LLMs if they are untrained. Models trained with \textbf{Case2Code} improve performance not only on distribution case-to-code induction but also on various coding-generation tasks, demonstrating the great potential of large-scale synthetic data and inductive learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_12504 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Case2Code: Scalable Synthetic Data for Code Generation Shao, Yunfan Li, Linyang Ma, Yichuan Li, Peiji Song, Demin Cheng, Qinyuan Li, Shimin Li, Xiaonan Wang, Pengyu Guo, Qipeng Yan, Hang Qiu, Xipeng Huang, Xuanjing Lin, Dahua Computation and Language Large Language Models (LLMs) have shown outstanding breakthroughs in code generation. Recent work improves code LLMs by training on synthetic data generated by some powerful LLMs, which can be challenging to scale due to the dependence on a teacher model and high generation costs. In this paper, we focus on synthesizing code data at scale and propose a \textbf{Case2Code} task by exploiting the expressiveness and correctness of programs. \textbf{Case2Code} is an inductive inference task that aims to infer underlying code implementations by observing input-output examples or program behaviors, By incorporating LLMs to generate program inputs, and executing the program with these inputs to obtain the program outputs, we can synthesize diverse and high-quality \textbf{Case2Code} data at scale for training and evaluating code LLMs. Experimental results show that case-to-code induction is challenging for current representative LLMs if they are untrained. Models trained with \textbf{Case2Code} improve performance not only on distribution case-to-code induction but also on various coding-generation tasks, demonstrating the great potential of large-scale synthetic data and inductive learning. |
| title | Case2Code: Scalable Synthetic Data for Code Generation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2407.12504 |