A Unit-based System and Dataset for Expressive Direct Speech-to-Speech Translation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916593557045248 |
|---|---|
| author | Min, Anna Hu, Chenxu Ren, Yi Zhao, Hang |
| author_facet | Min, Anna Hu, Chenxu Ren, Yi Zhao, Hang |
| contents | Current research in speech-to-speech translation (S2ST) primarily concentrates on translation accuracy and speech naturalness, often overlooking key elements like paralinguistic information, which is essential for conveying emotions and attitudes in communication. To address this, our research introduces a novel, carefully curated multilingual dataset from various movie audio tracks. Each dataset pair is precisely matched for paralinguistic information and duration. We enhance this by integrating multiple prosody transfer techniques, aiming for translations that are accurate, natural-sounding, and rich in paralinguistic details. Our experimental results confirm that our model retains more paralinguistic information from the source speech while maintaining high standards of translation accuracy and naturalness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_00374 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Unit-based System and Dataset for Expressive Direct Speech-to-Speech Translation Min, Anna Hu, Chenxu Ren, Yi Zhao, Hang Computation and Language Computer Vision and Pattern Recognition Multimedia Sound Audio and Speech Processing Current research in speech-to-speech translation (S2ST) primarily concentrates on translation accuracy and speech naturalness, often overlooking key elements like paralinguistic information, which is essential for conveying emotions and attitudes in communication. To address this, our research introduces a novel, carefully curated multilingual dataset from various movie audio tracks. Each dataset pair is precisely matched for paralinguistic information and duration. We enhance this by integrating multiple prosody transfer techniques, aiming for translations that are accurate, natural-sounding, and rich in paralinguistic details. Our experimental results confirm that our model retains more paralinguistic information from the source speech while maintaining high standards of translation accuracy and naturalness. |
| title | A Unit-based System and Dataset for Expressive Direct Speech-to-Speech Translation |
| topic | Computation and Language Computer Vision and Pattern Recognition Multimedia Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2502.00374 |