MOSS-Speech: Towards True Speech-to-Speech Models Without Text Guidance
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908572748611584 |
|---|---|
| author | Zhao, Xingjian Xu, Zhe Cheng, Qinyuan Fei, Zhaoye Jin, Luozhijie Wang, Yang Chen, Hanfu Jiang, Yaozhou Gao, Qinghui Chen, Ke Li, Ruixiao Chen, Mingshu Wang, Ruiming Zhang, Wenbo Zhang, Yiyang Yu, Donghua Gao, Yang Yang, Xiaogui Gong, Yitian Xu, Yuanfan Zhou, Yaqian Huang, Xuanjing Qiu, Xipeng |
| author_facet | Zhao, Xingjian Xu, Zhe Cheng, Qinyuan Fei, Zhaoye Jin, Luozhijie Wang, Yang Chen, Hanfu Jiang, Yaozhou Gao, Qinghui Chen, Ke Li, Ruixiao Chen, Mingshu Wang, Ruiming Zhang, Wenbo Zhang, Yiyang Yu, Donghua Gao, Yang Yang, Xiaogui Gong, Yitian Xu, Yuanfan Zhou, Yaqian Huang, Xuanjing Qiu, Xipeng |
| contents | Spoken dialogue systems often rely on cascaded pipelines that transcribe, process, and resynthesize speech. While effective, this design discards paralinguistic cues and limits expressivity. Recent end-to-end methods reduce latency and better preserve these cues, yet still rely on text intermediates, creating a fundamental bottleneck. We present MOSS-Speech, a true speech-to-speech large language model that directly understands and generates speech without relying on text guidance. Our approach combines a modality-based layer-splitting architecture with a frozen pre-training strategy, preserving the reasoning and knowledge of pretrained text LLMs while adding native speech capabilities. Experiments show that our model achieves state-of-the-art results in spoken question answering and delivers comparable speech-to-speech performance relative to existing text-guided systems, while still maintaining competitive text performance. By narrowing the gap between text-guided and direct speech generation, our work establishes a new paradigm for expressive and efficient end-to-end speech interaction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_00499 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MOSS-Speech: Towards True Speech-to-Speech Models Without Text Guidance Zhao, Xingjian Xu, Zhe Cheng, Qinyuan Fei, Zhaoye Jin, Luozhijie Wang, Yang Chen, Hanfu Jiang, Yaozhou Gao, Qinghui Chen, Ke Li, Ruixiao Chen, Mingshu Wang, Ruiming Zhang, Wenbo Zhang, Yiyang Yu, Donghua Gao, Yang Yang, Xiaogui Gong, Yitian Xu, Yuanfan Zhou, Yaqian Huang, Xuanjing Qiu, Xipeng Computation and Language Artificial Intelligence Spoken dialogue systems often rely on cascaded pipelines that transcribe, process, and resynthesize speech. While effective, this design discards paralinguistic cues and limits expressivity. Recent end-to-end methods reduce latency and better preserve these cues, yet still rely on text intermediates, creating a fundamental bottleneck. We present MOSS-Speech, a true speech-to-speech large language model that directly understands and generates speech without relying on text guidance. Our approach combines a modality-based layer-splitting architecture with a frozen pre-training strategy, preserving the reasoning and knowledge of pretrained text LLMs while adding native speech capabilities. Experiments show that our model achieves state-of-the-art results in spoken question answering and delivers comparable speech-to-speech performance relative to existing text-guided systems, while still maintaining competitive text performance. By narrowing the gap between text-guided and direct speech generation, our work establishes a new paradigm for expressive and efficient end-to-end speech interaction. |
| title | MOSS-Speech: Towards True Speech-to-Speech Models Without Text Guidance |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.00499 |