Qwen2.5-Coder Technical Report
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913576278556672 |
|---|---|
| author | Hui, Binyuan Yang, Jian Cui, Zeyu Yang, Jiaxi Liu, Dayiheng Zhang, Lei Liu, Tianyu Zhang, Jiajun Yu, Bowen Lu, Keming Dang, Kai Fan, Yang Zhang, Yichang Yang, An Men, Rui Huang, Fei Zheng, Bo Miao, Yibo Quan, Shanghaoran Feng, Yunlong Ren, Xingzhang Ren, Xuancheng Zhou, Jingren Lin, Junyang |
| author_facet | Hui, Binyuan Yang, Jian Cui, Zeyu Yang, Jiaxi Liu, Dayiheng Zhang, Lei Liu, Tianyu Zhang, Jiajun Yu, Bowen Lu, Keming Dang, Kai Fan, Yang Zhang, Yichang Yang, An Men, Rui Huang, Fei Zheng, Bo Miao, Yibo Quan, Shanghaoran Feng, Yunlong Ren, Xingzhang Ren, Xuancheng Zhou, Jingren Lin, Junyang |
| contents | In this report, we introduce the Qwen2.5-Coder series, a significant upgrade from its predecessor, CodeQwen1.5. This series includes six models: Qwen2.5-Coder-(0.5B/1.5B/3B/7B/14B/32B). As a code-specific model, Qwen2.5-Coder is built upon the Qwen2.5 architecture and continues pretrained on a vast corpus of over 5.5 trillion tokens. Through meticulous data cleaning, scalable synthetic data generation, and balanced data mixing, Qwen2.5-Coder demonstrates impressive code generation capabilities while retaining general and math skills. These models have been evaluated on a wide range of code-related tasks, achieving state-of-the-art (SOTA) performance across more than 10 benchmarks, including code generation, completion, reasoning, and repair, consistently outperforming larger models of the same model size. We believe that the release of the Qwen2.5-Coder series will advance research in code intelligence and, with its permissive licensing, support wider adoption by developers in real-world applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_12186 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Qwen2.5-Coder Technical Report Hui, Binyuan Yang, Jian Cui, Zeyu Yang, Jiaxi Liu, Dayiheng Zhang, Lei Liu, Tianyu Zhang, Jiajun Yu, Bowen Lu, Keming Dang, Kai Fan, Yang Zhang, Yichang Yang, An Men, Rui Huang, Fei Zheng, Bo Miao, Yibo Quan, Shanghaoran Feng, Yunlong Ren, Xingzhang Ren, Xuancheng Zhou, Jingren Lin, Junyang Computation and Language In this report, we introduce the Qwen2.5-Coder series, a significant upgrade from its predecessor, CodeQwen1.5. This series includes six models: Qwen2.5-Coder-(0.5B/1.5B/3B/7B/14B/32B). As a code-specific model, Qwen2.5-Coder is built upon the Qwen2.5 architecture and continues pretrained on a vast corpus of over 5.5 trillion tokens. Through meticulous data cleaning, scalable synthetic data generation, and balanced data mixing, Qwen2.5-Coder demonstrates impressive code generation capabilities while retaining general and math skills. These models have been evaluated on a wide range of code-related tasks, achieving state-of-the-art (SOTA) performance across more than 10 benchmarks, including code generation, completion, reasoning, and repair, consistently outperforming larger models of the same model size. We believe that the release of the Qwen2.5-Coder series will advance research in code intelligence and, with its permissive licensing, support wider adoption by developers in real-world applications. |
| title | Qwen2.5-Coder Technical Report |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.12186 |