OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913633561214976 |
|---|---|
| author | Cheng, Xize Fu, Dongjie Yang, Xiaoda Fang, Minghui Hu, Ruofan Lu, Jingyu Jionghao, Bai Wang, Zehan Ji, Shengpeng Huang, Rongjie Li, Linjun Chen, Yu Jin, Tao Zhao, Zhou |
| author_facet | Cheng, Xize Fu, Dongjie Yang, Xiaoda Fang, Minghui Hu, Ruofan Lu, Jingyu Jionghao, Bai Wang, Zehan Ji, Shengpeng Huang, Rongjie Li, Linjun Chen, Yu Jin, Tao Zhao, Zhou |
| contents | With the rapid development of large language models, researchers have created increasingly advanced spoken dialogue systems that can naturally converse with humans. However, these systems still struggle to handle the full complexity of real-world conversations, including audio events, musical contexts, and emotional expressions, mainly because current dialogue datasets are constrained in both scale and scenario diversity. In this paper, we propose leveraging synthetic data to enhance the dialogue models across diverse scenarios. We introduce ShareChatX, the first comprehensive, large-scale dataset for spoken dialogue that spans diverse scenarios. Based on this dataset, we introduce OmniChat, a multi-turn dialogue system with a heterogeneous feature fusion module, designed to optimize feature selection in different dialogue contexts. In addition, we explored critical aspects of training dialogue systems using synthetic data. Through comprehensive experimentation, we determined the ideal balance between synthetic and real data, achieving state-of-the-art results on the real-world dialogue dataset DailyTalk. We also highlight the crucial importance of synthetic data in tackling diverse, complex dialogue scenarios, especially those involving audio and music. For more details, please visit our demo page at \url{https://sharechatx.github.io/}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_01384 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios Cheng, Xize Fu, Dongjie Yang, Xiaoda Fang, Minghui Hu, Ruofan Lu, Jingyu Jionghao, Bai Wang, Zehan Ji, Shengpeng Huang, Rongjie Li, Linjun Chen, Yu Jin, Tao Zhao, Zhou Computation and Language Human-Computer Interaction Sound Audio and Speech Processing With the rapid development of large language models, researchers have created increasingly advanced spoken dialogue systems that can naturally converse with humans. However, these systems still struggle to handle the full complexity of real-world conversations, including audio events, musical contexts, and emotional expressions, mainly because current dialogue datasets are constrained in both scale and scenario diversity. In this paper, we propose leveraging synthetic data to enhance the dialogue models across diverse scenarios. We introduce ShareChatX, the first comprehensive, large-scale dataset for spoken dialogue that spans diverse scenarios. Based on this dataset, we introduce OmniChat, a multi-turn dialogue system with a heterogeneous feature fusion module, designed to optimize feature selection in different dialogue contexts. In addition, we explored critical aspects of training dialogue systems using synthetic data. Through comprehensive experimentation, we determined the ideal balance between synthetic and real data, achieving state-of-the-art results on the real-world dialogue dataset DailyTalk. We also highlight the crucial importance of synthetic data in tackling diverse, complex dialogue scenarios, especially those involving audio and music. For more details, please visit our demo page at \url{https://sharechatx.github.io/}. |
| title | OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios |
| topic | Computation and Language Human-Computer Interaction Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2501.01384 |