WizardCoder: Empowering Code Large Language Models with Evol-Instruct
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915306299981824 |
|---|---|
| author | Luo, Ziyang Xu, Can Zhao, Pu Sun, Qingfeng Geng, Xiubo Hu, Wenxiang Tao, Chongyang Ma, Jing Lin, Qingwei Jiang, Daxin |
| author_facet | Luo, Ziyang Xu, Can Zhao, Pu Sun, Qingfeng Geng, Xiubo Hu, Wenxiang Tao, Chongyang Ma, Jing Lin, Qingwei Jiang, Daxin |
| contents | Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. However, most existing models are solely pre-trained on extensive raw code data without instruction fine-tuning. In this paper, we introduce WizardCoder, which empowers Code LLMs with complex instruction fine-tuning, by adapting the Evol-Instruct method to the domain of code. Through comprehensive experiments on four prominent code generation benchmarks, namely HumanEval, HumanEval+, MBPP, and DS-1000, we unveil the exceptional capabilities of our model. It surpasses all other open-source Code LLMs by a substantial margin. Moreover, our model even outperforms the largest closed LLMs, Anthropic's Claude and Google's Bard, on HumanEval and HumanEval+. Our code, model weights, and data are public at https://github.com/nlpxucan/WizardLM |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_08568 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | WizardCoder: Empowering Code Large Language Models with Evol-Instruct Luo, Ziyang Xu, Can Zhao, Pu Sun, Qingfeng Geng, Xiubo Hu, Wenxiang Tao, Chongyang Ma, Jing Lin, Qingwei Jiang, Daxin Computation and Language Artificial Intelligence Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. However, most existing models are solely pre-trained on extensive raw code data without instruction fine-tuning. In this paper, we introduce WizardCoder, which empowers Code LLMs with complex instruction fine-tuning, by adapting the Evol-Instruct method to the domain of code. Through comprehensive experiments on four prominent code generation benchmarks, namely HumanEval, HumanEval+, MBPP, and DS-1000, we unveil the exceptional capabilities of our model. It surpasses all other open-source Code LLMs by a substantial margin. Moreover, our model even outperforms the largest closed LLMs, Anthropic's Claude and Google's Bard, on HumanEval and HumanEval+. Our code, model weights, and data are public at https://github.com/nlpxucan/WizardLM |
| title | WizardCoder: Empowering Code Large Language Models with Evol-Instruct |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2306.08568 |