Multi-Source Diffusion Models for Simultaneous Music Generation and Separation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929279957204992 |
|---|---|
| author | Mariani, Giorgio Tallini, Irene Postolache, Emilian Mancusi, Michele Cosmo, Luca Rodolà, Emanuele |
| author_facet | Mariani, Giorgio Tallini, Irene Postolache, Emilian Mancusi, Michele Cosmo, Luca Rodolà, Emanuele |
| contents | In this work, we define a diffusion-based generative model capable of both music synthesis and source separation by learning the score of the joint probability density of sources sharing a context. Alongside the classic total inference tasks (i.e., generating a mixture, separating the sources), we also introduce and experiment on the partial generation task of source imputation, where we generate a subset of the sources given the others (e.g., play a piano track that goes well with the drums). Additionally, we introduce a novel inference method for the separation task based on Dirac likelihood functions. We train our model on Slakh2100, a standard dataset for musical source separation, provide qualitative results in the generation settings, and showcase competitive quantitative results in the source separation setting. Our method is the first example of a single model that can handle both generation and separation tasks, thus representing a step toward general audio models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_02257 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Multi-Source Diffusion Models for Simultaneous Music Generation and Separation Mariani, Giorgio Tallini, Irene Postolache, Emilian Mancusi, Michele Cosmo, Luca Rodolà, Emanuele Sound Machine Learning Audio and Speech Processing In this work, we define a diffusion-based generative model capable of both music synthesis and source separation by learning the score of the joint probability density of sources sharing a context. Alongside the classic total inference tasks (i.e., generating a mixture, separating the sources), we also introduce and experiment on the partial generation task of source imputation, where we generate a subset of the sources given the others (e.g., play a piano track that goes well with the drums). Additionally, we introduce a novel inference method for the separation task based on Dirac likelihood functions. We train our model on Slakh2100, a standard dataset for musical source separation, provide qualitative results in the generation settings, and showcase competitive quantitative results in the source separation setting. Our method is the first example of a single model that can handle both generation and separation tasks, thus representing a step toward general audio models. |
| title | Multi-Source Diffusion Models for Simultaneous Music Generation and Separation |
| topic | Sound Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2302.02257 |