Audio-conditioned phonemic and prosodic annotation for building text-to-speech models from unlabeled speech data
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917691626881024 |
|---|---|
| author | Shirahata, Yuma Park, Byeongseon Yamamoto, Ryuichi Tachibana, Kentaro |
| author_facet | Shirahata, Yuma Park, Byeongseon Yamamoto, Ryuichi Tachibana, Kentaro |
| contents | This paper proposes an audio-conditioned phonemic and prosodic annotation model for building text-to-speech (TTS) datasets from unlabeled speech samples. For creating a TTS dataset that consists of label-speech paired data, the proposed annotation model leverages an automatic speech recognition (ASR) model to obtain phonemic and prosodic labels from unlabeled speech samples. By fine-tuning a large-scale pre-trained ASR model, we can construct the annotation model using a limited amount of label-speech paired data within an existing TTS dataset. To alleviate the shortage of label-speech paired data for training the annotation model, we generate pseudo label-speech paired data using text-only corpora and an auxiliary TTS model. This TTS model is also trained with the existing TTS dataset. Experimental results show that the TTS model trained with the dataset created by the proposed annotation method can synthesize speech as naturally as the one trained with a fully-labeled dataset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_08111 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Audio-conditioned phonemic and prosodic annotation for building text-to-speech models from unlabeled speech data Shirahata, Yuma Park, Byeongseon Yamamoto, Ryuichi Tachibana, Kentaro Audio and Speech Processing This paper proposes an audio-conditioned phonemic and prosodic annotation model for building text-to-speech (TTS) datasets from unlabeled speech samples. For creating a TTS dataset that consists of label-speech paired data, the proposed annotation model leverages an automatic speech recognition (ASR) model to obtain phonemic and prosodic labels from unlabeled speech samples. By fine-tuning a large-scale pre-trained ASR model, we can construct the annotation model using a limited amount of label-speech paired data within an existing TTS dataset. To alleviate the shortage of label-speech paired data for training the annotation model, we generate pseudo label-speech paired data using text-only corpora and an auxiliary TTS model. This TTS model is also trained with the existing TTS dataset. Experimental results show that the TTS model trained with the dataset created by the proposed annotation method can synthesize speech as naturally as the one trained with a fully-labeled dataset. |
| title | Audio-conditioned phonemic and prosodic annotation for building text-to-speech models from unlabeled speech data |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2406.08111 |