SoulX-Transcriber: A Robust End-to-End Framework for Multi-Speaker Speech Transcription
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916075023630336 |
|---|---|
| author | Dai, Yuhang Lin, Haopeng Lin, Zhennan Qian, Jiale Wu, Jun Xie, Hanke Meng, Hao Wen, Hanlin Ding, Chuang Yin, Shunshun Tao, Ming Xie, Lei Wang, Xinsheng |
| author_facet | Dai, Yuhang Lin, Haopeng Lin, Zhennan Qian, Jiale Wu, Jun Xie, Hanke Meng, Hao Wen, Hanlin Ding, Chuang Yin, Shunshun Tao, Ming Xie, Lei Wang, Xinsheng |
| contents | Recent advances in Automatic Speech Recognition (ASR) and Large Language Models (LLMs) have significantly improved speech understanding capabilities. However, multi-speaker speech transcription remains challenging task, constrained by highly similar speaker voices, rapid turn-taking transitions, overlapping utterances and inaccurate speaker boundary segmentation. These challenges become particularly pronounced in real-world conversational audio, where speaker dynamics and acoustic conditions are highly variable. This technical report presents SoulX-Transcriber, a unified multi-speaker transcription system that jointly models speaker diarization (SD) and ASR within an LLM-based framework. SoulX-Transcriber adopts a two-stage training strategy to improve both speaker discrimination and transcription robustness. In the first stage, speaker-aware multi-task continuous pre-training enhances speaker representation learning and boundary perception. In the second stage, supervised fine-tuning further optimizes the model for accurate end-to-end speaker-attributed transcription under complex multi-speaker conditions. SoulX-Transcriber delivers strong performance and robustness across multiple public benchmarks, including AliMeeting, AISHELL-4, and AMI, while maintaining high adaptability to multi-domain scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_02400 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SoulX-Transcriber: A Robust End-to-End Framework for Multi-Speaker Speech Transcription Dai, Yuhang Lin, Haopeng Lin, Zhennan Qian, Jiale Wu, Jun Xie, Hanke Meng, Hao Wen, Hanlin Ding, Chuang Yin, Shunshun Tao, Ming Xie, Lei Wang, Xinsheng Audio and Speech Processing Recent advances in Automatic Speech Recognition (ASR) and Large Language Models (LLMs) have significantly improved speech understanding capabilities. However, multi-speaker speech transcription remains challenging task, constrained by highly similar speaker voices, rapid turn-taking transitions, overlapping utterances and inaccurate speaker boundary segmentation. These challenges become particularly pronounced in real-world conversational audio, where speaker dynamics and acoustic conditions are highly variable. This technical report presents SoulX-Transcriber, a unified multi-speaker transcription system that jointly models speaker diarization (SD) and ASR within an LLM-based framework. SoulX-Transcriber adopts a two-stage training strategy to improve both speaker discrimination and transcription robustness. In the first stage, speaker-aware multi-task continuous pre-training enhances speaker representation learning and boundary perception. In the second stage, supervised fine-tuning further optimizes the model for accurate end-to-end speaker-attributed transcription under complex multi-speaker conditions. SoulX-Transcriber delivers strong performance and robustness across multiple public benchmarks, including AliMeeting, AISHELL-4, and AMI, while maintaining high adaptability to multi-domain scenarios. |
| title | SoulX-Transcriber: A Robust End-to-End Framework for Multi-Speaker Speech Transcription |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2606.02400 |