Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2605.23463 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910247738671104 |
|---|---|
| author | Lin, Bin Zhao, Bo Wu, Boyong Yan, Chao Wu, Chen Yi, Cheng Yao, Chengyuan Liu, Daijiao Tian, Fei Tian, Feng Sun, Haiyang Zhang, Haoyang Zhen, Jiangjie Gong, Jinglan Chen, Jun Xie, Li Li, Peilin Yang, Peng Tan, Pengfei Lin, Qingjian Li, Runze Hu, Shenghua Zhou, Siyi Qu, Wenwen Li, Xiangyu Zhang, Xiangyu Tony Yang, Xuerui Yang, Yang Huang, Yechang Fu, Yu Luo, Yuchu Li, Yuxin Zhang, Yuxin Sheng, Zhengyan Li, Brian Zeng, Chang Zhang, Changlin Geng, Chen Dong, Chenghao Feng, Chengli Zhou, Dan Wan, Danni Chen, Di Zhang, Die Pang, Dongqing Yang, Guanglong Hu, Guoqiang Zhu, Huangxi Gao, Jianzheng Liang, Jinghua Wan, Jinmei Yuan, Junjie An, Kang Lei, Lei Zhong, Limin Cai, Lun Ren, Mengqiang Xu, Min Li, Mingliang Li, Mingxiao Wang, Na Tong, Qiang Huang, Qiaoling Du, Qingfu Wang, Rui Zhou, Shengchen Qiu, Shi Peng, Shihao Yang, Shiliang Tu, Siqi Deng, Tianjiao Xu, Ting Wang, Tong Niu, WeiMing Xie, Wuxun Zhang, Xianwei Feng, Xianyu Liu, Xiaojia Chen, Xing Wu, Xiongbin Wu, Yan Li, Yang Liu, Yi Zhang, Yifan Liu, Yile Long, Yongshen Luo, Yu Ding, Yuanhao Wang, Yuhao Yin, Yuhe Xu, Yunfang Yang, Yuxiang Huang, Zhiguo Wu, Zhiyue Li, Zichao Zhou, Zichao Jiang, Daxin Li, Future Yu, Gang Zhang, Xiangyu Zhu, Yibo |
| author_facet | Lin, Bin Zhao, Bo Wu, Boyong Yan, Chao Wu, Chen Yi, Cheng Yao, Chengyuan Liu, Daijiao Tian, Fei Tian, Feng Sun, Haiyang Zhang, Haoyang Zhen, Jiangjie Gong, Jinglan Chen, Jun Xie, Li Li, Peilin Yang, Peng Tan, Pengfei Lin, Qingjian Li, Runze Hu, Shenghua Zhou, Siyi Qu, Wenwen Li, Xiangyu Zhang, Xiangyu Tony Yang, Xuerui Yang, Yang Huang, Yechang Fu, Yu Luo, Yuchu Li, Yuxin Zhang, Yuxin Sheng, Zhengyan Li, Brian Zeng, Chang Zhang, Changlin Geng, Chen Dong, Chenghao Feng, Chengli Zhou, Dan Wan, Danni Chen, Di Zhang, Die Pang, Dongqing Yang, Guanglong Hu, Guoqiang Zhu, Huangxi Gao, Jianzheng Liang, Jinghua Wan, Jinmei Yuan, Junjie An, Kang Lei, Lei Zhong, Limin Cai, Lun Ren, Mengqiang Xu, Min Li, Mingliang Li, Mingxiao Wang, Na Tong, Qiang Huang, Qiaoling Du, Qingfu Wang, Rui Zhou, Shengchen Qiu, Shi Peng, Shihao Yang, Shiliang Tu, Siqi Deng, Tianjiao Xu, Ting Wang, Tong Niu, WeiMing Xie, Wuxun Zhang, Xianwei Feng, Xianyu Liu, Xiaojia Chen, Xing Wu, Xiongbin Wu, Yan Li, Yang Liu, Yi Zhang, Yifan Liu, Yile Long, Yongshen Luo, Yu Ding, Yuanhao Wang, Yuhao Yin, Yuhe Xu, Yunfang Yang, Yuxiang Huang, Zhiguo Wu, Zhiyue Li, Zichao Zhou, Zichao Jiang, Daxin Li, Future Yu, Gang Zhang, Xiangyu Zhu, Yibo |
| contents | Unified audio-language modeling has emerged as a prominent trend in modern speech systems, promising to bring the reasoning capabilities of large language models to auditory tasks. However, existing unified foundations often struggle to match the depth of specialized systems across automatic speech recognition (ASR), text-to-speech synthesis (TTS), and realtime spoken interaction. Bridging this gap remains an open challenge. This report presents StepAudio 2.5, a unified audio-language foundation model that matches or exceeds specialized systems across all three capabilities. Rather than treating these tasks as architecturally distinct, we operate on the premise that once text and audio share a multimodal representational space, task specialization becomes a matter of operational regimes: data construction, optimization targets, and decoding constraints. Guided by this insight, we advance the post-training paradigm from standard supervised learning to task-tailored Reinforcement Learning from Human Feedback (RLHF), using it as the primary mechanism to define complex optimization targets. We leverage this RLHF-centric alignment, alongside specialized decoding, to shape a shared backbone into three distinct operational modes. Concretely, the ASR branch advances transcription efficiency via verifiable multi-token decoding; the TTS branch achieves controllable, expressive synthesis through preference-based RLHF and context-rich supervision; and the Realtime branch realizes low-latency, persona-consistent dialogue via generative reward modeling within an RLHF framework. On standard benchmarks, StepAudio 2.5 achieves state-of-the-art results across ASR, TTS, and Realtime, demonstrating that a singular audio-language foundation can successfully internalize the distinct deployment objectives of speech understanding, generation, and live interaction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_23463 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | StepAudio 2.5 Technical Report Lin, Bin Zhao, Bo Wu, Boyong Yan, Chao Wu, Chen Yi, Cheng Yao, Chengyuan Liu, Daijiao Tian, Fei Tian, Feng Sun, Haiyang Zhang, Haoyang Zhen, Jiangjie Gong, Jinglan Chen, Jun Xie, Li Li, Peilin Yang, Peng Tan, Pengfei Lin, Qingjian Li, Runze Hu, Shenghua Zhou, Siyi Qu, Wenwen Li, Xiangyu Zhang, Xiangyu Tony Yang, Xuerui Yang, Yang Huang, Yechang Fu, Yu Luo, Yuchu Li, Yuxin Zhang, Yuxin Sheng, Zhengyan Li, Brian Zeng, Chang Zhang, Changlin Geng, Chen Dong, Chenghao Feng, Chengli Zhou, Dan Wan, Danni Chen, Di Zhang, Die Pang, Dongqing Yang, Guanglong Hu, Guoqiang Zhu, Huangxi Gao, Jianzheng Liang, Jinghua Wan, Jinmei Yuan, Junjie An, Kang Lei, Lei Zhong, Limin Cai, Lun Ren, Mengqiang Xu, Min Li, Mingliang Li, Mingxiao Wang, Na Tong, Qiang Huang, Qiaoling Du, Qingfu Wang, Rui Zhou, Shengchen Qiu, Shi Peng, Shihao Yang, Shiliang Tu, Siqi Deng, Tianjiao Xu, Ting Wang, Tong Niu, WeiMing Xie, Wuxun Zhang, Xianwei Feng, Xianyu Liu, Xiaojia Chen, Xing Wu, Xiongbin Wu, Yan Li, Yang Liu, Yi Zhang, Yifan Liu, Yile Long, Yongshen Luo, Yu Ding, Yuanhao Wang, Yuhao Yin, Yuhe Xu, Yunfang Yang, Yuxiang Huang, Zhiguo Wu, Zhiyue Li, Zichao Zhou, Zichao Jiang, Daxin Li, Future Yu, Gang Zhang, Xiangyu Zhu, Yibo Audio and Speech Processing Unified audio-language modeling has emerged as a prominent trend in modern speech systems, promising to bring the reasoning capabilities of large language models to auditory tasks. However, existing unified foundations often struggle to match the depth of specialized systems across automatic speech recognition (ASR), text-to-speech synthesis (TTS), and realtime spoken interaction. Bridging this gap remains an open challenge. This report presents StepAudio 2.5, a unified audio-language foundation model that matches or exceeds specialized systems across all three capabilities. Rather than treating these tasks as architecturally distinct, we operate on the premise that once text and audio share a multimodal representational space, task specialization becomes a matter of operational regimes: data construction, optimization targets, and decoding constraints. Guided by this insight, we advance the post-training paradigm from standard supervised learning to task-tailored Reinforcement Learning from Human Feedback (RLHF), using it as the primary mechanism to define complex optimization targets. We leverage this RLHF-centric alignment, alongside specialized decoding, to shape a shared backbone into three distinct operational modes. Concretely, the ASR branch advances transcription efficiency via verifiable multi-token decoding; the TTS branch achieves controllable, expressive synthesis through preference-based RLHF and context-rich supervision; and the Realtime branch realizes low-latency, persona-consistent dialogue via generative reward modeling within an RLHF framework. On standard benchmarks, StepAudio 2.5 achieves state-of-the-art results across ASR, TTS, and Realtime, demonstrating that a singular audio-language foundation can successfully internalize the distinct deployment objectives of speech understanding, generation, and live interaction. |
| title | StepAudio 2.5 Technical Report |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2605.23463 |