LLM-Enhanced Dialogue Management for Full-Duplex Spoken Dialogue Systems
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910843053015040 |
|---|---|
| author | Zhang, Hao Li, Weiwei Chen, Rilin Kothapally, Vinay Yu, Meng Yu, Dong |
| author_facet | Zhang, Hao Li, Weiwei Chen, Rilin Kothapally, Vinay Yu, Meng Yu, Dong |
| contents | Achieving full-duplex communication in spoken dialogue systems (SDS) requires real-time coordination between listening, speaking, and thinking. This paper proposes a semantic voice activity detection (VAD) module as a dialogue manager (DM) to efficiently manage turn-taking in full-duplex SDS. Implemented as a lightweight (0.5B) LLM fine-tuned on full-duplex conversation data, the semantic VAD predicts four control tokens to regulate turn-switching and turn-keeping, distinguishing between intentional and unintentional barge-ins while detecting query completion for handling user pauses and hesitations. By processing input speech in short intervals, the semantic VAD enables real-time decision-making, while the core dialogue engine (CDE) is only activated for response generation, reducing computational overhead. This design allows independent DM optimization without retraining the CDE, balancing interaction accuracy and inference efficiency for scalable, next-generation full-duplex SDS. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_14145 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLM-Enhanced Dialogue Management for Full-Duplex Spoken Dialogue Systems Zhang, Hao Li, Weiwei Chen, Rilin Kothapally, Vinay Yu, Meng Yu, Dong Computation and Language Audio and Speech Processing Achieving full-duplex communication in spoken dialogue systems (SDS) requires real-time coordination between listening, speaking, and thinking. This paper proposes a semantic voice activity detection (VAD) module as a dialogue manager (DM) to efficiently manage turn-taking in full-duplex SDS. Implemented as a lightweight (0.5B) LLM fine-tuned on full-duplex conversation data, the semantic VAD predicts four control tokens to regulate turn-switching and turn-keeping, distinguishing between intentional and unintentional barge-ins while detecting query completion for handling user pauses and hesitations. By processing input speech in short intervals, the semantic VAD enables real-time decision-making, while the core dialogue engine (CDE) is only activated for response generation, reducing computational overhead. This design allows independent DM optimization without retraining the CDE, balancing interaction accuracy and inference efficiency for scalable, next-generation full-duplex SDS. |
| title | LLM-Enhanced Dialogue Management for Full-Duplex Spoken Dialogue Systems |
| topic | Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2502.14145 |