Streaming Speaker Change Detection and Gender Classification for Transducer-Based Multi-Talker Speech Translation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916597907587072 |
|---|---|
| author | Wang, Peidong Kanda, Naoyuki Xue, Jian Li, Jinyu Wang, Xiaofei Subramanian, Aswin Shanmugam Chen, Junkun Sivasankaran, Sunit Xiao, Xiong Zhao, Yong |
| author_facet | Wang, Peidong Kanda, Naoyuki Xue, Jian Li, Jinyu Wang, Xiaofei Subramanian, Aswin Shanmugam Chen, Junkun Sivasankaran, Sunit Xiao, Xiong Zhao, Yong |
| contents | Streaming multi-talker speech translation is a task that involves not only generating accurate and fluent translations with low latency but also recognizing when a speaker change occurs and what the speaker's gender is. Speaker change information can be used to create audio prompts for a zero-shot text-to-speech system, and gender can help to select speaker profiles in a conventional text-to-speech model. We propose to tackle streaming speaker change detection and gender classification by incorporating speaker embeddings into a transducer-based streaming end-to-end speech translation model. Our experiments demonstrate that the proposed methods can achieve high accuracy for both speaker change detection and gender classification. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_02683 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Streaming Speaker Change Detection and Gender Classification for Transducer-Based Multi-Talker Speech Translation Wang, Peidong Kanda, Naoyuki Xue, Jian Li, Jinyu Wang, Xiaofei Subramanian, Aswin Shanmugam Chen, Junkun Sivasankaran, Sunit Xiao, Xiong Zhao, Yong Sound Artificial Intelligence Computation and Language Audio and Speech Processing Streaming multi-talker speech translation is a task that involves not only generating accurate and fluent translations with low latency but also recognizing when a speaker change occurs and what the speaker's gender is. Speaker change information can be used to create audio prompts for a zero-shot text-to-speech system, and gender can help to select speaker profiles in a conventional text-to-speech model. We propose to tackle streaming speaker change detection and gender classification by incorporating speaker embeddings into a transducer-based streaming end-to-end speech translation model. Our experiments demonstrate that the proposed methods can achieve high accuracy for both speaker change detection and gender classification. |
| title | Streaming Speaker Change Detection and Gender Classification for Transducer-Based Multi-Talker Speech Translation |
| topic | Sound Artificial Intelligence Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2502.02683 |