Debatts: Zero-Shot Debating Text-to-Speech Synthesis
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909414828539904 |
|---|---|
| author | Huang, Yiqiao Wang, Yuancheng Li, Jiaqi Guo, Haotian He, Haorui Zhang, Shunsi Wu, Zhizheng |
| author_facet | Huang, Yiqiao Wang, Yuancheng Li, Jiaqi Guo, Haotian He, Haorui Zhang, Shunsi Wu, Zhizheng |
| contents | In debating, rebuttal is one of the most critical stages, where a speaker addresses the arguments presented by the opposing side. During this process, the speaker synthesizes their own persuasive articulation given the context from the opposing side. This work proposes a novel zero-shot text-to-speech synthesis system for rebuttal, namely Debatts. Debatts takes two speech prompts, one from the opposing side (i.e. opponent) and one from the speaker. The prompt from the opponent is supposed to provide debating style prosody, and the prompt from the speaker provides identity information. In particular, we pretrain the Debatts system from in-the-wild dataset, and integrate an additional reference encoder to take debating prompt for style. In addition, we also create a debating dataset to develop Debatts. In this setting, Debatts can generate a debating-style speech in rebuttal for any voices. Experimental results confirm the effectiveness of the proposed system in comparison with the classic zero-shot TTS systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_06540 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Debatts: Zero-Shot Debating Text-to-Speech Synthesis Huang, Yiqiao Wang, Yuancheng Li, Jiaqi Guo, Haotian He, Haorui Zhang, Shunsi Wu, Zhizheng Audio and Speech Processing In debating, rebuttal is one of the most critical stages, where a speaker addresses the arguments presented by the opposing side. During this process, the speaker synthesizes their own persuasive articulation given the context from the opposing side. This work proposes a novel zero-shot text-to-speech synthesis system for rebuttal, namely Debatts. Debatts takes two speech prompts, one from the opposing side (i.e. opponent) and one from the speaker. The prompt from the opponent is supposed to provide debating style prosody, and the prompt from the speaker provides identity information. In particular, we pretrain the Debatts system from in-the-wild dataset, and integrate an additional reference encoder to take debating prompt for style. In addition, we also create a debating dataset to develop Debatts. In this setting, Debatts can generate a debating-style speech in rebuttal for any voices. Experimental results confirm the effectiveness of the proposed system in comparison with the classic zero-shot TTS systems. |
| title | Debatts: Zero-Shot Debating Text-to-Speech Synthesis |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2411.06540 |