Accent conversion using discrete units with parallel data synthesized from controllable accented TTS
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914965286289408 |
|---|---|
| author | Nguyen, Tuan Nam Pham, Ngoc Quan Waibel, Alexander |
| author_facet | Nguyen, Tuan Nam Pham, Ngoc Quan Waibel, Alexander |
| contents | The goal of accent conversion (AC) is to convert speech accents while preserving content and speaker identity. Previous methods either required reference utterances during inference, did not preserve speaker identity well, or used one-to-one systems that could only be trained for each non-native accent. This paper presents a promising AC model that can convert many accents into native to overcome these issues. Our approach utilizes discrete units, derived from clustering self-supervised representations of native speech, as an intermediary target for accent conversion. Leveraging multi-speaker text-to-speech synthesis, it transforms these discrete representations back into native speech while retaining the speaker identity. Additionally, we develop an efficient data augmentation method to train the system without demanding a lot of non-native resources. Our system is proved to improve non-native speaker fluency, sound like a native accent, and preserve original speaker identity well. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_03734 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Accent conversion using discrete units with parallel data synthesized from controllable accented TTS Nguyen, Tuan Nam Pham, Ngoc Quan Waibel, Alexander Sound Computation and Language Audio and Speech Processing The goal of accent conversion (AC) is to convert speech accents while preserving content and speaker identity. Previous methods either required reference utterances during inference, did not preserve speaker identity well, or used one-to-one systems that could only be trained for each non-native accent. This paper presents a promising AC model that can convert many accents into native to overcome these issues. Our approach utilizes discrete units, derived from clustering self-supervised representations of native speech, as an intermediary target for accent conversion. Leveraging multi-speaker text-to-speech synthesis, it transforms these discrete representations back into native speech while retaining the speaker identity. Additionally, we develop an efficient data augmentation method to train the system without demanding a lot of non-native resources. Our system is proved to improve non-native speaker fluency, sound like a native accent, and preserve original speaker identity well. |
| title | Accent conversion using discrete units with parallel data synthesized from controllable accented TTS |
| topic | Sound Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2410.03734 |