TokSing: Singing Voice Synthesis based on Discrete Tokens
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913397986033664 |
|---|---|
| author | Wu, Yuning zhang, Chunlei Shi, Jiatong Tang, Yuxun Yang, Shan Jin, Qin |
| author_facet | Wu, Yuning zhang, Chunlei Shi, Jiatong Tang, Yuxun Yang, Shan Jin, Qin |
| contents | Recent advancements in speech synthesis witness significant benefits by leveraging discrete tokens extracted from self-supervised learning (SSL) models. Discrete tokens offer higher storage efficiency and greater operability in intermediate representations compared to traditional continuous Mel spectrograms. However, when it comes to singing voice synthesis(SVS), achieving higher levels of melody expression poses a great challenge for utilizing discrete tokens. In this paper, we introduce TokSing, a discrete-based SVS system equipped with a token formulator that offers flexible token blendings. We observe a melody degradation during discretization, prompting us to integrate a melody signal with the discrete token and incorporate a specially-designed melody enhancement strategy in the musical encoder. Extensive experiments demonstrate that our TokSing achieves better performance against the Mel spectrogram baselines while offering advantages in intermediate representation space cost and convergence speed. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_08416 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TokSing: Singing Voice Synthesis based on Discrete Tokens Wu, Yuning zhang, Chunlei Shi, Jiatong Tang, Yuxun Yang, Shan Jin, Qin Sound Audio and Speech Processing Recent advancements in speech synthesis witness significant benefits by leveraging discrete tokens extracted from self-supervised learning (SSL) models. Discrete tokens offer higher storage efficiency and greater operability in intermediate representations compared to traditional continuous Mel spectrograms. However, when it comes to singing voice synthesis(SVS), achieving higher levels of melody expression poses a great challenge for utilizing discrete tokens. In this paper, we introduce TokSing, a discrete-based SVS system equipped with a token formulator that offers flexible token blendings. We observe a melody degradation during discretization, prompting us to integrate a melody signal with the discrete token and incorporate a specially-designed melody enhancement strategy in the musical encoder. Extensive experiments demonstrate that our TokSing achieves better performance against the Mel spectrogram baselines while offering advantages in intermediate representation space cost and convergence speed. |
| title | TokSing: Singing Voice Synthesis based on Discrete Tokens |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2406.08416 |