FastTurn: Unifying Acoustic and Streaming Semantic Cues for Low-Latency and Robust Turn Detection
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908994849734656 |
|---|---|
| author | Wang, Chengyou Xue, Hongfei He, Chunjiang Hu, Jingbin Wang, Shuiyuan Wu, Bo Ji, Yuyu Zheng, Jimeng Chen, Ruofei Zhu, Zhou Xie, Lei |
| author_facet | Wang, Chengyou Xue, Hongfei He, Chunjiang Hu, Jingbin Wang, Shuiyuan Wu, Bo Ji, Yuyu Zheng, Jimeng Chen, Ruofei Zhu, Zhou Xie, Lei |
| contents | Recent advances in AudioLLMs have enabled spoken dialogue systems to move beyond turn-based interaction toward real-time full-duplex communication, where the agent must decide when to speak, yield, or interrupt while the user is still talking. Existing full-duplex approaches either rely on voice activity cues, which lack semantic understanding, or on ASR-based modules, which introduce latency and degrade under overlapping speech and noise. Moreover, available datasets rarely capture realistic interaction dynamics, limiting evaluation and deployment. To mitigate the problem, we propose \textbf{FastTurn}, a unified framework for low-latency and robust turn detection. To advance latency while maintaining performance, FastTurn combines streaming CTC decoding with acoustic features, enabling early decisions from partial observations while preserving semantic cues. We also release a test set based on real human dialogue, capturing authentic turn transitions, overlapping speech, backchannels, pauses, pitch variation, and environmental noise. Experiments show FastTurn achieves higher decision accuracy with lower interruption latency than representative baselines and remains robust under challenging acoustic conditions, demonstrating its effectiveness for practical full-duplex dialogue systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_01897 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | FastTurn: Unifying Acoustic and Streaming Semantic Cues for Low-Latency and Robust Turn Detection Wang, Chengyou Xue, Hongfei He, Chunjiang Hu, Jingbin Wang, Shuiyuan Wu, Bo Ji, Yuyu Zheng, Jimeng Chen, Ruofei Zhu, Zhou Xie, Lei Sound Audio and Speech Processing Recent advances in AudioLLMs have enabled spoken dialogue systems to move beyond turn-based interaction toward real-time full-duplex communication, where the agent must decide when to speak, yield, or interrupt while the user is still talking. Existing full-duplex approaches either rely on voice activity cues, which lack semantic understanding, or on ASR-based modules, which introduce latency and degrade under overlapping speech and noise. Moreover, available datasets rarely capture realistic interaction dynamics, limiting evaluation and deployment. To mitigate the problem, we propose \textbf{FastTurn}, a unified framework for low-latency and robust turn detection. To advance latency while maintaining performance, FastTurn combines streaming CTC decoding with acoustic features, enabling early decisions from partial observations while preserving semantic cues. We also release a test set based on real human dialogue, capturing authentic turn transitions, overlapping speech, backchannels, pauses, pitch variation, and environmental noise. Experiments show FastTurn achieves higher decision accuracy with lower interruption latency than representative baselines and remains robust under challenging acoustic conditions, demonstrating its effectiveness for practical full-duplex dialogue systems. |
| title | FastTurn: Unifying Acoustic and Streaming Semantic Cues for Low-Latency and Robust Turn Detection |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2604.01897 |