ESPnet-SDS: Unified Toolkit and Demo for Spoken Dialogue Systems
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912270035976192 |
|---|---|
| author | Arora, Siddhant Peng, Yifan Shi, Jiatong Tian, Jinchuan Chen, William Bharadwaj, Shikhar Futami, Hayato Kashiwagi, Yosuke Tsunoo, Emiru Shimizu, Shuichiro Srivastav, Vaibhav Watanabe, Shinji |
| author_facet | Arora, Siddhant Peng, Yifan Shi, Jiatong Tian, Jinchuan Chen, William Bharadwaj, Shikhar Futami, Hayato Kashiwagi, Yosuke Tsunoo, Emiru Shimizu, Shuichiro Srivastav, Vaibhav Watanabe, Shinji |
| contents | Advancements in audio foundation models (FMs) have fueled interest in end-to-end (E2E) spoken dialogue systems, but different web interfaces for each system makes it challenging to compare and contrast them effectively. Motivated by this, we introduce an open-source, user-friendly toolkit designed to build unified web interfaces for various cascaded and E2E spoken dialogue systems. Our demo further provides users with the option to get on-the-fly automated evaluation metrics such as (1) latency, (2) ability to understand user input, (3) coherence, diversity, and relevance of system response, and (4) intelligibility and audio quality of system output. Using the evaluation metrics, we compare various cascaded and E2E spoken dialogue systems with a human-human conversation dataset as a proxy. Our analysis demonstrates that the toolkit allows researchers to effortlessly compare and contrast different technologies, providing valuable insights such as current E2E systems having poorer audio quality and less diverse responses. An example demo produced using our toolkit is publicly available here: https://huggingface.co/spaces/Siddhant/Voice_Assistant_Demo. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_08533 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ESPnet-SDS: Unified Toolkit and Demo for Spoken Dialogue Systems Arora, Siddhant Peng, Yifan Shi, Jiatong Tian, Jinchuan Chen, William Bharadwaj, Shikhar Futami, Hayato Kashiwagi, Yosuke Tsunoo, Emiru Shimizu, Shuichiro Srivastav, Vaibhav Watanabe, Shinji Computation and Language Sound Audio and Speech Processing Advancements in audio foundation models (FMs) have fueled interest in end-to-end (E2E) spoken dialogue systems, but different web interfaces for each system makes it challenging to compare and contrast them effectively. Motivated by this, we introduce an open-source, user-friendly toolkit designed to build unified web interfaces for various cascaded and E2E spoken dialogue systems. Our demo further provides users with the option to get on-the-fly automated evaluation metrics such as (1) latency, (2) ability to understand user input, (3) coherence, diversity, and relevance of system response, and (4) intelligibility and audio quality of system output. Using the evaluation metrics, we compare various cascaded and E2E spoken dialogue systems with a human-human conversation dataset as a proxy. Our analysis demonstrates that the toolkit allows researchers to effortlessly compare and contrast different technologies, providing valuable insights such as current E2E systems having poorer audio quality and less diverse responses. An example demo produced using our toolkit is publicly available here: https://huggingface.co/spaces/Siddhant/Voice_Assistant_Demo. |
| title | ESPnet-SDS: Unified Toolkit and Demo for Spoken Dialogue Systems |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2503.08533 |