StemGen: A music generation model that listens
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916091389804544 |
|---|---|
| author | Parker, Julian D. Spijkervet, Janne Kosta, Katerina Yesiler, Furkan Kuznetsov, Boris Wang, Ju-Chiang Avent, Matt Chen, Jitong Le, Duc |
| author_facet | Parker, Julian D. Spijkervet, Janne Kosta, Katerina Yesiler, Furkan Kuznetsov, Boris Wang, Ju-Chiang Avent, Matt Chen, Jitong Le, Duc |
| contents | End-to-end generation of musical audio using deep learning techniques has seen an explosion of activity recently. However, most models concentrate on generating fully mixed music in response to abstract conditioning information. In this work, we present an alternative paradigm for producing music generation models that can listen and respond to musical context. We describe how such a model can be constructed using a non-autoregressive, transformer-based model architecture and present a number of novel architectural and sampling improvements. We train the described architecture on both an open-source and a proprietary dataset. We evaluate the produced models using standard quality metrics and a new approach based on music information retrieval descriptors. The resulting model reaches the audio quality of state-of-the-art text-conditioned models, as well as exhibiting strong musical coherence with its context. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_08723 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | StemGen: A music generation model that listens Parker, Julian D. Spijkervet, Janne Kosta, Katerina Yesiler, Furkan Kuznetsov, Boris Wang, Ju-Chiang Avent, Matt Chen, Jitong Le, Duc Sound Machine Learning Audio and Speech Processing End-to-end generation of musical audio using deep learning techniques has seen an explosion of activity recently. However, most models concentrate on generating fully mixed music in response to abstract conditioning information. In this work, we present an alternative paradigm for producing music generation models that can listen and respond to musical context. We describe how such a model can be constructed using a non-autoregressive, transformer-based model architecture and present a number of novel architectural and sampling improvements. We train the described architecture on both an open-source and a proprietary dataset. We evaluate the produced models using standard quality metrics and a new approach based on music information retrieval descriptors. The resulting model reaches the audio quality of state-of-the-art text-conditioned models, as well as exhibiting strong musical coherence with its context. |
| title | StemGen: A music generation model that listens |
| topic | Sound Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2312.08723 |