Muskits-ESPnet: A Comprehensive Toolkit for Singing Voice Synthesis in New Paradigm
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912068882399232 |
|---|---|
| author | Wu, Yuning Shi, Jiatong Yu, Yifeng Tang, Yuxun Qian, Tao Lin, Yueqian Han, Jionghao Bai, Xinyi Watanabe, Shinji Jin, Qin |
| author_facet | Wu, Yuning Shi, Jiatong Yu, Yifeng Tang, Yuxun Qian, Tao Lin, Yueqian Han, Jionghao Bai, Xinyi Watanabe, Shinji Jin, Qin |
| contents | This research presents Muskits-ESPnet, a versatile toolkit that introduces new paradigms to Singing Voice Synthesis (SVS) through the application of pretrained audio models in both continuous and discrete approaches. Specifically, we explore discrete representations derived from SSL models and audio codecs and offer significant advantages in versatility and intelligence, supporting multi-format inputs and adaptable data processing workflows for various SVS models. The toolkit features automatic music score error detection and correction, as well as a perception auto-evaluation module to imitate human subjective evaluating scores. Muskits-ESPnet is available at \url{https://github.com/espnet/espnet}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_07226 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Muskits-ESPnet: A Comprehensive Toolkit for Singing Voice Synthesis in New Paradigm Wu, Yuning Shi, Jiatong Yu, Yifeng Tang, Yuxun Qian, Tao Lin, Yueqian Han, Jionghao Bai, Xinyi Watanabe, Shinji Jin, Qin Sound Audio and Speech Processing This research presents Muskits-ESPnet, a versatile toolkit that introduces new paradigms to Singing Voice Synthesis (SVS) through the application of pretrained audio models in both continuous and discrete approaches. Specifically, we explore discrete representations derived from SSL models and audio codecs and offer significant advantages in versatility and intelligence, supporting multi-format inputs and adaptable data processing workflows for various SVS models. The toolkit features automatic music score error detection and correction, as well as a perception auto-evaluation module to imitate human subjective evaluating scores. Muskits-ESPnet is available at \url{https://github.com/espnet/espnet}. |
| title | Muskits-ESPnet: A Comprehensive Toolkit for Singing Voice Synthesis in New Paradigm |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2409.07226 |