SingingSDS: A Singing-Capable Spoken Dialogue System for Conversational Roleplay Applications
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911336824307712 |
|---|---|
| author | Han, Jionghao Shi, Jiatong Someki, Masao Tang, Yuxun Liu, Lan Zhao, Yiwen Feng, Wenhao Watanabe, Shinji |
| author_facet | Han, Jionghao Shi, Jiatong Someki, Masao Tang, Yuxun Liu, Lan Zhao, Yiwen Feng, Wenhao Watanabe, Shinji |
| contents | With recent advances in automatic speech recognition (ASR), large language models (LLMs), and text-to-speech (TTS) technologies, spoken dialogue systems (SDS) have become widely accessible. However, most existing SDS are limited to conventional spoken responses. We present SingingSDS, a cascaded SDS that responds through singing rather than speaking, fostering more affective, memorable, and pleasurable interactions in character-based roleplay and interactive entertainment scenarios. SingingSDS employs a modular ASR-LLM-SVS pipeline and supports a wide range of configurations across character personas, ASR and LLM backends, SVS models, melody sources, and voice profiles, tailored to different needs in terms of latency, quality, and musical style. SingingSDS is available as a plug-and-play web demo, featuring modular, open-source code that supports customization and extension. Demo: https://huggingface.co/spaces/espnet/SingingSDS. Code: https://github.com/SingingSDS/SingingSDS. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_20972 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SingingSDS: A Singing-Capable Spoken Dialogue System for Conversational Roleplay Applications Han, Jionghao Shi, Jiatong Someki, Masao Tang, Yuxun Liu, Lan Zhao, Yiwen Feng, Wenhao Watanabe, Shinji Sound With recent advances in automatic speech recognition (ASR), large language models (LLMs), and text-to-speech (TTS) technologies, spoken dialogue systems (SDS) have become widely accessible. However, most existing SDS are limited to conventional spoken responses. We present SingingSDS, a cascaded SDS that responds through singing rather than speaking, fostering more affective, memorable, and pleasurable interactions in character-based roleplay and interactive entertainment scenarios. SingingSDS employs a modular ASR-LLM-SVS pipeline and supports a wide range of configurations across character personas, ASR and LLM backends, SVS models, melody sources, and voice profiles, tailored to different needs in terms of latency, quality, and musical style. SingingSDS is available as a plug-and-play web demo, featuring modular, open-source code that supports customization and extension. Demo: https://huggingface.co/spaces/espnet/SingingSDS. Code: https://github.com/SingingSDS/SingingSDS. |
| title | SingingSDS: A Singing-Capable Spoken Dialogue System for Conversational Roleplay Applications |
| topic | Sound |
| url | https://arxiv.org/abs/2511.20972 |