Music Style Transfer with Time-Varying Inversion of Diffusion Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913238515449856 |
|---|---|
| author | Li, Sifei Zhang, Yuxin Tang, Fan Ma, Chongyang dong, Weiming Xu, Changsheng |
| author_facet | Li, Sifei Zhang, Yuxin Tang, Fan Ma, Chongyang dong, Weiming Xu, Changsheng |
| contents | With the development of diffusion models, text-guided image style transfer has demonstrated high-quality controllable synthesis results. However, the utilization of text for diverse music style transfer poses significant challenges, primarily due to the limited availability of matched audio-text datasets. Music, being an abstract and complex art form, exhibits variations and intricacies even within the same genre, thereby making accurate textual descriptions challenging. This paper presents a music style transfer approach that effectively captures musical attributes using minimal data. We introduce a novel time-varying textual inversion module to precisely capture mel-spectrogram features at different levels. During inference, we propose a bias-reduced stylization technique to obtain stable results. Experimental results demonstrate that our method can transfer the style of specific instruments, as well as incorporate natural sounds to compose melodies. Samples and source code are available at https://lsfhuihuiff.github.io/MusicTI/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_13763 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Music Style Transfer with Time-Varying Inversion of Diffusion Models Li, Sifei Zhang, Yuxin Tang, Fan Ma, Chongyang dong, Weiming Xu, Changsheng Sound Audio and Speech Processing With the development of diffusion models, text-guided image style transfer has demonstrated high-quality controllable synthesis results. However, the utilization of text for diverse music style transfer poses significant challenges, primarily due to the limited availability of matched audio-text datasets. Music, being an abstract and complex art form, exhibits variations and intricacies even within the same genre, thereby making accurate textual descriptions challenging. This paper presents a music style transfer approach that effectively captures musical attributes using minimal data. We introduce a novel time-varying textual inversion module to precisely capture mel-spectrogram features at different levels. During inference, we propose a bias-reduced stylization technique to obtain stable results. Experimental results demonstrate that our method can transfer the style of specific instruments, as well as incorporate natural sounds to compose melodies. Samples and source code are available at https://lsfhuihuiff.github.io/MusicTI/. |
| title | Music Style Transfer with Time-Varying Inversion of Diffusion Models |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2402.13763 |