SymPAC: Scalable Symbolic Music Generation With Prompts And Constraints
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913494310322176 |
|---|---|
| author | Chen, Haonan Smith, Jordan B. L. Spijkervet, Janne Wang, Ju-Chiang Zou, Pei Li, Bochen Kong, Qiuqiang Du, Xingjian |
| author_facet | Chen, Haonan Smith, Jordan B. L. Spijkervet, Janne Wang, Ju-Chiang Zou, Pei Li, Bochen Kong, Qiuqiang Du, Xingjian |
| contents | Progress in the task of symbolic music generation may be lagging behind other tasks like audio and text generation, in part because of the scarcity of symbolic training data. In this paper, we leverage the greater scale of audio music data by applying pre-trained MIR models (for transcription, beat tracking, structure analysis, etc.) to extract symbolic events and encode them into token sequences. To the best of our knowledge, this work is the first to demonstrate the feasibility of training symbolic generation models solely from auto-transcribed audio data. Furthermore, to enhance the controllability of the trained model, we introduce SymPAC (Symbolic Music Language Model with Prompting And Constrained Generation), which is distinguished by using (a) prompt bars in encoding and (b) a technique called Constrained Generation via Finite State Machines (FSMs) during inference time. We show the flexibility and controllability of this approach, which may be critical in making music AI useful to creators and users. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_03055 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SymPAC: Scalable Symbolic Music Generation With Prompts And Constraints Chen, Haonan Smith, Jordan B. L. Spijkervet, Janne Wang, Ju-Chiang Zou, Pei Li, Bochen Kong, Qiuqiang Du, Xingjian Sound Audio and Speech Processing Progress in the task of symbolic music generation may be lagging behind other tasks like audio and text generation, in part because of the scarcity of symbolic training data. In this paper, we leverage the greater scale of audio music data by applying pre-trained MIR models (for transcription, beat tracking, structure analysis, etc.) to extract symbolic events and encode them into token sequences. To the best of our knowledge, this work is the first to demonstrate the feasibility of training symbolic generation models solely from auto-transcribed audio data. Furthermore, to enhance the controllability of the trained model, we introduce SymPAC (Symbolic Music Language Model with Prompting And Constrained Generation), which is distinguished by using (a) prompt bars in encoding and (b) a technique called Constrained Generation via Finite State Machines (FSMs) during inference time. We show the flexibility and controllability of this approach, which may be critical in making music AI useful to creators and users. |
| title | SymPAC: Scalable Symbolic Music Generation With Prompts And Constraints |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2409.03055 |