Enabling Beam Search for Language Model-Based Text-to-Speech Synthesis
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910581885239296 |
|---|---|
| author | Tu, Zehai Zhang, Guangyan Lu, Yiting Adigwe, Adaeze King, Simon Guo, Yiwen |
| author_facet | Tu, Zehai Zhang, Guangyan Lu, Yiting Adigwe, Adaeze King, Simon Guo, Yiwen |
| contents | Tokenising continuous speech into sequences of discrete tokens and modelling them with language models (LMs) has led to significant success in text-to-speech (TTS) synthesis. Although these models can generate speech with high quality and naturalness, their synthesised samples can still suffer from artefacts, mispronunciation, word repeating, etc. In this paper, we argue these undesirable properties could partly be caused by the randomness of sampling-based strategies during the autoregressive decoding of LMs. Therefore, we look at maximisation-based decoding approaches and propose Temporal Repetition Aware Diverse Beam Search (TRAD-BS) to find the most probable sequences of the generated speech tokens. Experiments with two state-of-the-art LM-based TTS models demonstrate that our proposed maximisation-based decoding strategy generates speech with fewer mispronunciations and improved speaker consistency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_16373 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Enabling Beam Search for Language Model-Based Text-to-Speech Synthesis Tu, Zehai Zhang, Guangyan Lu, Yiting Adigwe, Adaeze King, Simon Guo, Yiwen Sound Audio and Speech Processing Tokenising continuous speech into sequences of discrete tokens and modelling them with language models (LMs) has led to significant success in text-to-speech (TTS) synthesis. Although these models can generate speech with high quality and naturalness, their synthesised samples can still suffer from artefacts, mispronunciation, word repeating, etc. In this paper, we argue these undesirable properties could partly be caused by the randomness of sampling-based strategies during the autoregressive decoding of LMs. Therefore, we look at maximisation-based decoding approaches and propose Temporal Repetition Aware Diverse Beam Search (TRAD-BS) to find the most probable sequences of the generated speech tokens. Experiments with two state-of-the-art LM-based TTS models demonstrate that our proposed maximisation-based decoding strategy generates speech with fewer mispronunciations and improved speaker consistency. |
| title | Enabling Beam Search for Language Model-Based Text-to-Speech Synthesis |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2408.16373 |