Cover Image

Step-Audio 2 Technical Report

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wu, Boyong, Yan, Chao, Hu, Chen, Yi, Cheng, Feng, Chengli, Tian, Fei, Shen, Feiyu, Yu, Gang, Zhang, Haoyang, Li, Jingbei, Chen, Mingrui, Liu, Peng, You, Wang, Zhang, Xiangyu Tony, Li, Xingyuan, Yang, Xuerui, Deng, Yayue, Huang, Yechang, Li, Yuxin, Zhang, Yuxin, You, Zhao, Li, Brian, Wan, Changyi, Hu, Hanpeng, Zhen, Jiangjie, Chen, Siyu, Yuan, Song, Zhang, Xuelin, Jiang, Yimin, Zhou, Yu, Yang, Yuxiang, Li, Bingxin, Ma, Buyun, Song, Changhe, Pang, Dongqing, Hu, Guoqiang, Sun, Haiyang, An, Kang, Wang, Na, Gao, Shuli, Ji, Wei, Li, Wen, Sun, Wen, Wen, Xuan, Ren, Yong, Ma, Yuankai, Lu, Yufan, Wang, Bin, Li, Bo, Miao, Changxin, Liu, Che, Xu, Chen, Shi, Dapeng, Hu, Dingyuan, Wu, Donghang, Liu, Enle, Huang, Guanzhe, Yan, Gulin, Zhang, Han, Nie, Hao, Jia, Haonan, Zhou, Hongyu, Sun, Jianjian, Wu, Jiaoren, Wu, Jie, Yang, Jie, Yang, Jin, Lin, Junzhe, Li, Kaixiang, Yang, Lei, Shi, Liying, Zhou, Li, Gu, Longlong, Li, Ming, Li, Mingliang, Li, Mingxiao, Wu, Nan, Han, Qi, Tan, Qinyuan, Pang, Shaoliang, Fan, Shengjie, Liu, Siqi, Cao, Tiancheng, Lu, Wanying, He, Wenqing, Xie, Wuxun, Zhao, Xu, Li, Xueqi, Yu, Yanbo, Yang, Yang, Liu, Yi, Lu, Yifan, Wang, Yilei, Ding, Yuanhao, Liang, Yuanwei, Lu, Yuanwei, Luo, Yuchu, Yin, Yuhe, Zhan, Yumeng, Zhang, Yuxiang, Yang, Zidong, Zhang, Zixin, Jiao, Binxing, Jiang, Daxin, Shum, Heung-Yeung, Chen, Jiansheng, Li, Jing, Zhang, Xiangyu, Zhu, Yibo
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Sound Audio and Speech Processing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866918131501367296
author	Wu, Boyong Yan, Chao Hu, Chen Yi, Cheng Feng, Chengli Tian, Fei Shen, Feiyu Yu, Gang Zhang, Haoyang Li, Jingbei Chen, Mingrui Liu, Peng You, Wang Zhang, Xiangyu Tony Li, Xingyuan Yang, Xuerui Deng, Yayue Huang, Yechang Li, Yuxin Zhang, Yuxin You, Zhao Li, Brian Wan, Changyi Hu, Hanpeng Zhen, Jiangjie Chen, Siyu Yuan, Song Zhang, Xuelin Jiang, Yimin Zhou, Yu Yang, Yuxiang Li, Bingxin Ma, Buyun Song, Changhe Pang, Dongqing Hu, Guoqiang Sun, Haiyang An, Kang Wang, Na Gao, Shuli Ji, Wei Li, Wen Sun, Wen Wen, Xuan Ren, Yong Ma, Yuankai Lu, Yufan Wang, Bin Li, Bo Miao, Changxin Liu, Che Xu, Chen Shi, Dapeng Hu, Dingyuan Wu, Donghang Liu, Enle Huang, Guanzhe Yan, Gulin Zhang, Han Nie, Hao Jia, Haonan Zhou, Hongyu Sun, Jianjian Wu, Jiaoren Wu, Jie Yang, Jie Yang, Jin Lin, Junzhe Li, Kaixiang Yang, Lei Shi, Liying Zhou, Li Gu, Longlong Li, Ming Li, Mingliang Li, Mingxiao Wu, Nan Han, Qi Tan, Qinyuan Pang, Shaoliang Fan, Shengjie Liu, Siqi Cao, Tiancheng Lu, Wanying He, Wenqing Xie, Wuxun Zhao, Xu Li, Xueqi Yu, Yanbo Yang, Yang Liu, Yi Lu, Yifan Wang, Yilei Ding, Yuanhao Liang, Yuanwei Lu, Yuanwei Luo, Yuchu Yin, Yuhe Zhan, Yumeng Zhang, Yuxiang Yang, Zidong Zhang, Zixin Jiao, Binxing Jiang, Daxin Shum, Heung-Yeung Chen, Jiansheng Li, Jing Zhang, Xiangyu Zhu, Yibo
author_facet	Wu, Boyong Yan, Chao Hu, Chen Yi, Cheng Feng, Chengli Tian, Fei Shen, Feiyu Yu, Gang Zhang, Haoyang Li, Jingbei Chen, Mingrui Liu, Peng You, Wang Zhang, Xiangyu Tony Li, Xingyuan Yang, Xuerui Deng, Yayue Huang, Yechang Li, Yuxin Zhang, Yuxin You, Zhao Li, Brian Wan, Changyi Hu, Hanpeng Zhen, Jiangjie Chen, Siyu Yuan, Song Zhang, Xuelin Jiang, Yimin Zhou, Yu Yang, Yuxiang Li, Bingxin Ma, Buyun Song, Changhe Pang, Dongqing Hu, Guoqiang Sun, Haiyang An, Kang Wang, Na Gao, Shuli Ji, Wei Li, Wen Sun, Wen Wen, Xuan Ren, Yong Ma, Yuankai Lu, Yufan Wang, Bin Li, Bo Miao, Changxin Liu, Che Xu, Chen Shi, Dapeng Hu, Dingyuan Wu, Donghang Liu, Enle Huang, Guanzhe Yan, Gulin Zhang, Han Nie, Hao Jia, Haonan Zhou, Hongyu Sun, Jianjian Wu, Jiaoren Wu, Jie Yang, Jie Yang, Jin Lin, Junzhe Li, Kaixiang Yang, Lei Shi, Liying Zhou, Li Gu, Longlong Li, Ming Li, Mingliang Li, Mingxiao Wu, Nan Han, Qi Tan, Qinyuan Pang, Shaoliang Fan, Shengjie Liu, Siqi Cao, Tiancheng Lu, Wanying He, Wenqing Xie, Wuxun Zhao, Xu Li, Xueqi Yu, Yanbo Yang, Yang Liu, Yi Lu, Yifan Wang, Yilei Ding, Yuanhao Liang, Yuanwei Lu, Yuanwei Luo, Yuchu Yin, Yuhe Zhan, Yumeng Zhang, Yuxiang Yang, Zidong Zhang, Zixin Jiao, Binxing Jiang, Daxin Shum, Heung-Yeung Chen, Jiansheng Li, Jing Zhang, Xiangyu Zhu, Yibo
contents	This paper presents Step-Audio 2, an end-to-end multi-modal large language model designed for industry-strength audio understanding and speech conversation. By integrating a latent audio encoder and reasoning-centric reinforcement learning (RL), Step-Audio 2 achieves promising performance in automatic speech recognition (ASR) and audio understanding. To facilitate genuine end-to-end speech conversation, Step-Audio 2 incorporates the generation of discrete audio tokens into language modeling, significantly enhancing its responsiveness to paralinguistic information such as speaking styles and emotions. To effectively leverage the rich textual and acoustic knowledge in real-world data, Step-Audio 2 integrates retrieval-augmented generation (RAG) and is able to call external tools such as web search to mitigate hallucination and audio search to switch timbres. Trained on millions of hours of speech and audio data, Step-Audio 2 delivers intelligence and expressiveness across diverse conversational scenarios. Evaluation results demonstrate that Step-Audio 2 achieves state-of-the-art performance on various audio understanding and conversational benchmarks compared to other open-source and commercial solutions. Please visit https://github.com/stepfun-ai/Step-Audio2 for more information.
format	Preprint
id	arxiv_https___arxiv_org_abs_2507_16632
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Step-Audio 2 Technical Report Wu, Boyong Yan, Chao Hu, Chen Yi, Cheng Feng, Chengli Tian, Fei Shen, Feiyu Yu, Gang Zhang, Haoyang Li, Jingbei Chen, Mingrui Liu, Peng You, Wang Zhang, Xiangyu Tony Li, Xingyuan Yang, Xuerui Deng, Yayue Huang, Yechang Li, Yuxin Zhang, Yuxin You, Zhao Li, Brian Wan, Changyi Hu, Hanpeng Zhen, Jiangjie Chen, Siyu Yuan, Song Zhang, Xuelin Jiang, Yimin Zhou, Yu Yang, Yuxiang Li, Bingxin Ma, Buyun Song, Changhe Pang, Dongqing Hu, Guoqiang Sun, Haiyang An, Kang Wang, Na Gao, Shuli Ji, Wei Li, Wen Sun, Wen Wen, Xuan Ren, Yong Ma, Yuankai Lu, Yufan Wang, Bin Li, Bo Miao, Changxin Liu, Che Xu, Chen Shi, Dapeng Hu, Dingyuan Wu, Donghang Liu, Enle Huang, Guanzhe Yan, Gulin Zhang, Han Nie, Hao Jia, Haonan Zhou, Hongyu Sun, Jianjian Wu, Jiaoren Wu, Jie Yang, Jie Yang, Jin Lin, Junzhe Li, Kaixiang Yang, Lei Shi, Liying Zhou, Li Gu, Longlong Li, Ming Li, Mingliang Li, Mingxiao Wu, Nan Han, Qi Tan, Qinyuan Pang, Shaoliang Fan, Shengjie Liu, Siqi Cao, Tiancheng Lu, Wanying He, Wenqing Xie, Wuxun Zhao, Xu Li, Xueqi Yu, Yanbo Yang, Yang Liu, Yi Lu, Yifan Wang, Yilei Ding, Yuanhao Liang, Yuanwei Lu, Yuanwei Luo, Yuchu Yin, Yuhe Zhan, Yumeng Zhang, Yuxiang Yang, Zidong Zhang, Zixin Jiao, Binxing Jiang, Daxin Shum, Heung-Yeung Chen, Jiansheng Li, Jing Zhang, Xiangyu Zhu, Yibo Computation and Language Sound Audio and Speech Processing This paper presents Step-Audio 2, an end-to-end multi-modal large language model designed for industry-strength audio understanding and speech conversation. By integrating a latent audio encoder and reasoning-centric reinforcement learning (RL), Step-Audio 2 achieves promising performance in automatic speech recognition (ASR) and audio understanding. To facilitate genuine end-to-end speech conversation, Step-Audio 2 incorporates the generation of discrete audio tokens into language modeling, significantly enhancing its responsiveness to paralinguistic information such as speaking styles and emotions. To effectively leverage the rich textual and acoustic knowledge in real-world data, Step-Audio 2 integrates retrieval-augmented generation (RAG) and is able to call external tools such as web search to mitigate hallucination and audio search to switch timbres. Trained on millions of hours of speech and audio data, Step-Audio 2 delivers intelligence and expressiveness across diverse conversational scenarios. Evaluation results demonstrate that Step-Audio 2 achieves state-of-the-art performance on various audio understanding and conversational benchmarks compared to other open-source and commercial solutions. Please visit https://github.com/stepfun-ai/Step-Audio2 for more information.
title	Step-Audio 2 Technical Report
topic	Computation and Language Sound Audio and Speech Processing
url	https://arxiv.org/abs/2507.16632