DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909083356889088 |
|---|---|
| author | Guo, Daya Zhu, Qihao Yang, Dejian Xie, Zhenda Dong, Kai Zhang, Wentao Chen, Guanting Bi, Xiao Wu, Y. Li, Y. K. Luo, Fuli Xiong, Yingfei Liang, Wenfeng |
| author_facet | Guo, Daya Zhu, Qihao Yang, Dejian Xie, Zhenda Dong, Kai Zhang, Wentao Chen, Guanting Bi, Xiao Wu, Y. Li, Y. K. Luo, Fuli Xiong, Yingfei Liang, Wenfeng |
| contents | The rapid development of large language models has revolutionized code intelligence in software development. However, the predominance of closed-source models has restricted extensive research and development. To address this, we introduce the DeepSeek-Coder series, a range of open-source code models with sizes from 1.3B to 33B, trained from scratch on 2 trillion tokens. These models are pre-trained on a high-quality project-level code corpus and employ a fill-in-the-blank task with a 16K window to enhance code generation and infilling. Our extensive evaluations demonstrate that DeepSeek-Coder not only achieves state-of-the-art performance among open-source code models across multiple benchmarks but also surpasses existing closed-source models like Codex and GPT-3.5. Furthermore, DeepSeek-Coder models are under a permissive license that allows for both research and unrestricted commercial use. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_14196 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence Guo, Daya Zhu, Qihao Yang, Dejian Xie, Zhenda Dong, Kai Zhang, Wentao Chen, Guanting Bi, Xiao Wu, Y. Li, Y. K. Luo, Fuli Xiong, Yingfei Liang, Wenfeng Software Engineering Computation and Language Machine Learning The rapid development of large language models has revolutionized code intelligence in software development. However, the predominance of closed-source models has restricted extensive research and development. To address this, we introduce the DeepSeek-Coder series, a range of open-source code models with sizes from 1.3B to 33B, trained from scratch on 2 trillion tokens. These models are pre-trained on a high-quality project-level code corpus and employ a fill-in-the-blank task with a 16K window to enhance code generation and infilling. Our extensive evaluations demonstrate that DeepSeek-Coder not only achieves state-of-the-art performance among open-source code models across multiple benchmarks but also surpasses existing closed-source models like Codex and GPT-3.5. Furthermore, DeepSeek-Coder models are under a permissive license that allows for both research and unrestricted commercial use. |
| title | DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence |
| topic | Software Engineering Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2401.14196 |