Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909371230846976 |
|---|---|
| author | Nistal, Javier Pasini, Marco Aouameur, Cyran Grachten, Maarten Lattner, Stefan |
| author_facet | Nistal, Javier Pasini, Marco Aouameur, Cyran Grachten, Maarten Lattner, Stefan |
| contents | Recent advancements in deep generative models present new opportunities for music production but also pose challenges, such as high computational demands and limited audio quality. Moreover, current systems frequently rely solely on text input and typically focus on producing complete musical pieces, which is incompatible with existing workflows in music production. To address these issues, we introduce "Diff-A-Riff," a Latent Diffusion Model designed to generate high-quality instrumental accompaniments adaptable to any musical context. This model offers control through either audio references, text prompts, or both, and produces 48kHz pseudo-stereo audio while significantly reducing inference time and memory usage. We demonstrate the model's capabilities through objective metrics and subjective listening tests, with extensive examples available on the accompanying website: sonycslparis.github.io/diffariff-companion/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_08384 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Models Nistal, Javier Pasini, Marco Aouameur, Cyran Grachten, Maarten Lattner, Stefan Sound Artificial Intelligence Audio and Speech Processing Recent advancements in deep generative models present new opportunities for music production but also pose challenges, such as high computational demands and limited audio quality. Moreover, current systems frequently rely solely on text input and typically focus on producing complete musical pieces, which is incompatible with existing workflows in music production. To address these issues, we introduce "Diff-A-Riff," a Latent Diffusion Model designed to generate high-quality instrumental accompaniments adaptable to any musical context. This model offers control through either audio references, text prompts, or both, and produces 48kHz pseudo-stereo audio while significantly reducing inference time and memory usage. We demonstrate the model's capabilities through objective metrics and subjective listening tests, with extensive examples available on the accompanying website: sonycslparis.github.io/diffariff-companion/ |
| title | Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Models |
| topic | Sound Artificial Intelligence Audio and Speech Processing |
| url | https://arxiv.org/abs/2406.08384 |