Qwen3-Omni Technical Report
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916960725368832 |
|---|---|
| author | Xu, Jin Guo, Zhifang Hu, Hangrui Chu, Yunfei Wang, Xiong He, Jinzheng Wang, Yuxuan Shi, Xian He, Ting Zhu, Xinfa Lv, Yuanjun Wang, Yongqi Guo, Dake Wang, He Ma, Linhan Zhang, Pei Zhang, Xinyu Hao, Hongkun Guo, Zishan Yang, Baosong Zhang, Bin Ma, Ziyang Wei, Xipin Bai, Shuai Chen, Keqin Liu, Xuejing Wang, Peng Yang, Mingkun Liu, Dayiheng Ren, Xingzhang Zheng, Bo Men, Rui Zhou, Fan Yu, Bowen Yang, Jianxin Yu, Le Zhou, Jingren Lin, Junyang |
| author_facet | Xu, Jin Guo, Zhifang Hu, Hangrui Chu, Yunfei Wang, Xiong He, Jinzheng Wang, Yuxuan Shi, Xian He, Ting Zhu, Xinfa Lv, Yuanjun Wang, Yongqi Guo, Dake Wang, He Ma, Linhan Zhang, Pei Zhang, Xinyu Hao, Hongkun Guo, Zishan Yang, Baosong Zhang, Bin Ma, Ziyang Wei, Xipin Bai, Shuai Chen, Keqin Liu, Xuejing Wang, Peng Yang, Mingkun Liu, Dayiheng Ren, Xingzhang Zheng, Bo Men, Rui Zhou, Fan Yu, Bowen Yang, Jianxin Yu, Le Zhou, Jingren Lin, Junyang |
| contents | We present Qwen3-Omni, a single multimodal model that, for the first time, maintains state-of-the-art performance across text, image, audio, and video without any degradation relative to single-modal counterparts. Qwen3-Omni matches the performance of same-sized single-modal models within the Qwen series and excels particularly on audio tasks. Across 36 audio and audio-visual benchmarks, Qwen3-Omni achieves open-source SOTA on 32 benchmarks and overall SOTA on 22, outperforming strong closed-source models such as Gemini-2.5-Pro, Seed-ASR, and GPT-4o-Transcribe. Qwen3-Omni adopts a Thinker-Talker MoE architecture that unifies perception and generation across text, images, audio, and video, yielding fluent text and natural real-time speech. It supports text interaction in 119 languages, speech understanding in 19 languages, and speech generation in 10 languages. To reduce first-packet latency in streaming synthesis, Talker autoregressively predicts discrete speech codecs using a multi-codebook scheme. Leveraging the representational capacity of these codebooks, we replace computationally intensive block-wise diffusion with a lightweight causal ConvNet, enabling streaming from the first codec frame. In cold-start settings, Qwen3-Omni achieves a theoretical end-to-end first-packet latency of 234 ms. To further strengthen multimodal reasoning, we introduce a Thinking model that explicitly reasons over inputs from any modality. Since the research community currently lacks a general-purpose audio captioning model, we fine-tuned Qwen3-Omni-30B-A3B to obtain Qwen3-Omni-30B-A3B-Captioner, which produces detailed, low-hallucination captions for arbitrary audio inputs. Qwen3-Omni-30B-A3B, Qwen3-Omni-30B-A3B-Thinking, and Qwen3-Omni-30B-A3B-Captioner are publicly released under the Apache 2.0 license. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_17765 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Qwen3-Omni Technical Report Xu, Jin Guo, Zhifang Hu, Hangrui Chu, Yunfei Wang, Xiong He, Jinzheng Wang, Yuxuan Shi, Xian He, Ting Zhu, Xinfa Lv, Yuanjun Wang, Yongqi Guo, Dake Wang, He Ma, Linhan Zhang, Pei Zhang, Xinyu Hao, Hongkun Guo, Zishan Yang, Baosong Zhang, Bin Ma, Ziyang Wei, Xipin Bai, Shuai Chen, Keqin Liu, Xuejing Wang, Peng Yang, Mingkun Liu, Dayiheng Ren, Xingzhang Zheng, Bo Men, Rui Zhou, Fan Yu, Bowen Yang, Jianxin Yu, Le Zhou, Jingren Lin, Junyang Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Audio and Speech Processing We present Qwen3-Omni, a single multimodal model that, for the first time, maintains state-of-the-art performance across text, image, audio, and video without any degradation relative to single-modal counterparts. Qwen3-Omni matches the performance of same-sized single-modal models within the Qwen series and excels particularly on audio tasks. Across 36 audio and audio-visual benchmarks, Qwen3-Omni achieves open-source SOTA on 32 benchmarks and overall SOTA on 22, outperforming strong closed-source models such as Gemini-2.5-Pro, Seed-ASR, and GPT-4o-Transcribe. Qwen3-Omni adopts a Thinker-Talker MoE architecture that unifies perception and generation across text, images, audio, and video, yielding fluent text and natural real-time speech. It supports text interaction in 119 languages, speech understanding in 19 languages, and speech generation in 10 languages. To reduce first-packet latency in streaming synthesis, Talker autoregressively predicts discrete speech codecs using a multi-codebook scheme. Leveraging the representational capacity of these codebooks, we replace computationally intensive block-wise diffusion with a lightweight causal ConvNet, enabling streaming from the first codec frame. In cold-start settings, Qwen3-Omni achieves a theoretical end-to-end first-packet latency of 234 ms. To further strengthen multimodal reasoning, we introduce a Thinking model that explicitly reasons over inputs from any modality. Since the research community currently lacks a general-purpose audio captioning model, we fine-tuned Qwen3-Omni-30B-A3B to obtain Qwen3-Omni-30B-A3B-Captioner, which produces detailed, low-hallucination captions for arbitrary audio inputs. Qwen3-Omni-30B-A3B, Qwen3-Omni-30B-A3B-Thinking, and Qwen3-Omni-30B-A3B-Captioner are publicly released under the Apache 2.0 license. |
| title | Qwen3-Omni Technical Report |
| topic | Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Audio and Speech Processing |
| url | https://arxiv.org/abs/2509.17765 |