MuCodec: Ultra Low-Bitrate Music Codec
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912476647391232 |
|---|---|
| author | Xu, Yaoxun Chen, Hangting Yu, Jianwei Tan, Wei Gu, Rongzhi Lei, Shun Lin, Zhiwei Wu, Zhiyong |
| author_facet | Xu, Yaoxun Chen, Hangting Yu, Jianwei Tan, Wei Gu, Rongzhi Lei, Shun Lin, Zhiwei Wu, Zhiyong |
| contents | Music codecs are a vital aspect of audio codec research, and ultra low-bitrate compression holds significant importance for music transmission and generation. Due to the complexity of music backgrounds and the richness of vocals, solely relying on modeling semantic or acoustic information cannot effectively reconstruct music with both vocals and backgrounds. To address this issue, we propose MuCodec, specifically targeting music compression and reconstruction tasks at ultra low bitrates. MuCodec employs MuEncoder to extract both acoustic and semantic features, discretizes them with RVQ, and obtains Mel-VAE features via flow-matching. The music is then reconstructed using a pre-trained MEL-VAE decoder and HiFi-GAN. MuCodec can reconstruct high-fidelity music at ultra low (0.35kbps) or high bitrates (1.35kbps), achieving the best results to date in both subjective and objective metrics. Code and Demo: https://xuyaoxun.github.io/MuCodec_demo/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_13216 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MuCodec: Ultra Low-Bitrate Music Codec Xu, Yaoxun Chen, Hangting Yu, Jianwei Tan, Wei Gu, Rongzhi Lei, Shun Lin, Zhiwei Wu, Zhiyong Sound Audio and Speech Processing Music codecs are a vital aspect of audio codec research, and ultra low-bitrate compression holds significant importance for music transmission and generation. Due to the complexity of music backgrounds and the richness of vocals, solely relying on modeling semantic or acoustic information cannot effectively reconstruct music with both vocals and backgrounds. To address this issue, we propose MuCodec, specifically targeting music compression and reconstruction tasks at ultra low bitrates. MuCodec employs MuEncoder to extract both acoustic and semantic features, discretizes them with RVQ, and obtains Mel-VAE features via flow-matching. The music is then reconstructed using a pre-trained MEL-VAE decoder and HiFi-GAN. MuCodec can reconstruct high-fidelity music at ultra low (0.35kbps) or high bitrates (1.35kbps), achieving the best results to date in both subjective and objective metrics. Code and Demo: https://xuyaoxun.github.io/MuCodec_demo/. |
| title | MuCodec: Ultra Low-Bitrate Music Codec |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2409.13216 |