Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.18013 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908469910568960 |
|---|---|
| author | Wang, Zihan Liu, Xinzhang Yao, Yitong Wang, Chao Zhao, Yu Yang, Zhihao Deng, Wenmin Jia, Kaipeng Peng, Jiaxin Huang, Yuyao Xiong, Sishi Jiang, Zhuo Yu, Kaidong Hu, Xiaohui Yao, Fubei Fang, Ruiyu Jiang, Zhuoru Song, Ruiting Xie, Qiyi Xue, Rui He, Xuewei Xue, Yanlei Yuan, Zhu Zhang, Zhaoxi Huang, Zilu Wang, Shiquan Wang, Xin Wu, Hanming Wang, Mingyuan Zhan, Xufeng Sun, Yuhan Xing, Zhaohu Jiang, Yuhao Yang, Bingkai Song, Shuangyong Li, Yongxiang He, Zhongjiang Li, Xuelong |
| author_facet | Wang, Zihan Liu, Xinzhang Yao, Yitong Wang, Chao Zhao, Yu Yang, Zhihao Deng, Wenmin Jia, Kaipeng Peng, Jiaxin Huang, Yuyao Xiong, Sishi Jiang, Zhuo Yu, Kaidong Hu, Xiaohui Yao, Fubei Fang, Ruiyu Jiang, Zhuoru Song, Ruiting Xie, Qiyi Xue, Rui He, Xuewei Xue, Yanlei Yuan, Zhu Zhang, Zhaoxi Huang, Zilu Wang, Shiquan Wang, Xin Wu, Hanming Wang, Mingyuan Zhan, Xufeng Sun, Yuhan Xing, Zhaohu Jiang, Yuhao Yang, Bingkai Song, Shuangyong Li, Yongxiang He, Zhongjiang Li, Xuelong |
| contents | We introduce the latest series of TeleChat models: \textbf{TeleChat2}, \textbf{TeleChat2.5}, and \textbf{T1}, offering a significant upgrade over their predecessor, TeleChat. Despite minimal changes to the model architecture, the new series achieves substantial performance gains through enhanced training strategies in both pre-training and post-training stages. The series begins with \textbf{TeleChat2}, which undergoes pretraining on 10 trillion high-quality and diverse tokens. This is followed by Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) to further enhance its capabilities. \textbf{TeleChat2.5} and \textbf{T1} expand the pipeline by incorporating a continual pretraining phase with domain-specific datasets, combined with reinforcement learning (RL) to improve performance in code generation and mathematical reasoning tasks. The \textbf{T1} variant is designed for complex reasoning, supporting long Chain-of-Thought (CoT) reasoning and demonstrating substantial improvements in mathematics and coding. In contrast, \textbf{TeleChat2.5} prioritizes speed, delivering rapid inference. Both flagship models of \textbf{T1} and \textbf{TeleChat2.5} are dense Transformer-based architectures with 115B parameters, showcasing significant advancements in reasoning and general task performance compared to the original TeleChat. Notably, \textbf{T1-115B} outperform proprietary models such as OpenAI's o1-mini and GPT-4o. We publicly release \textbf{TeleChat2}, \textbf{TeleChat2.5} and \textbf{T1}, including post-trained versions with 35B and 115B parameters, to empower developers and researchers with state-of-the-art language models tailored for diverse applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_18013 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Technical Report of TeleChat2, TeleChat2.5 and T1 Wang, Zihan Liu, Xinzhang Yao, Yitong Wang, Chao Zhao, Yu Yang, Zhihao Deng, Wenmin Jia, Kaipeng Peng, Jiaxin Huang, Yuyao Xiong, Sishi Jiang, Zhuo Yu, Kaidong Hu, Xiaohui Yao, Fubei Fang, Ruiyu Jiang, Zhuoru Song, Ruiting Xie, Qiyi Xue, Rui He, Xuewei Xue, Yanlei Yuan, Zhu Zhang, Zhaoxi Huang, Zilu Wang, Shiquan Wang, Xin Wu, Hanming Wang, Mingyuan Zhan, Xufeng Sun, Yuhan Xing, Zhaohu Jiang, Yuhao Yang, Bingkai Song, Shuangyong Li, Yongxiang He, Zhongjiang Li, Xuelong Computation and Language I.2.7 We introduce the latest series of TeleChat models: \textbf{TeleChat2}, \textbf{TeleChat2.5}, and \textbf{T1}, offering a significant upgrade over their predecessor, TeleChat. Despite minimal changes to the model architecture, the new series achieves substantial performance gains through enhanced training strategies in both pre-training and post-training stages. The series begins with \textbf{TeleChat2}, which undergoes pretraining on 10 trillion high-quality and diverse tokens. This is followed by Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) to further enhance its capabilities. \textbf{TeleChat2.5} and \textbf{T1} expand the pipeline by incorporating a continual pretraining phase with domain-specific datasets, combined with reinforcement learning (RL) to improve performance in code generation and mathematical reasoning tasks. The \textbf{T1} variant is designed for complex reasoning, supporting long Chain-of-Thought (CoT) reasoning and demonstrating substantial improvements in mathematics and coding. In contrast, \textbf{TeleChat2.5} prioritizes speed, delivering rapid inference. Both flagship models of \textbf{T1} and \textbf{TeleChat2.5} are dense Transformer-based architectures with 115B parameters, showcasing significant advancements in reasoning and general task performance compared to the original TeleChat. Notably, \textbf{T1-115B} outperform proprietary models such as OpenAI's o1-mini and GPT-4o. We publicly release \textbf{TeleChat2}, \textbf{TeleChat2.5} and \textbf{T1}, including post-trained versions with 35B and 115B parameters, to empower developers and researchers with state-of-the-art language models tailored for diverse applications. |
| title | Technical Report of TeleChat2, TeleChat2.5 and T1 |
| topic | Computation and Language I.2.7 |
| url | https://arxiv.org/abs/2507.18013 |