Nested Music Transformer: Sequentially Decoding Compound Tokens in Symbolic Music and Audio Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915863449305088 |
|---|---|
| author | Yoo, HaeJun Dong, Hao-Wen Jung, Jongmin Jeong, Dasaem |
| author_facet | Yoo, HaeJun Dong, Hao-Wen Jung, Jongmin Jeong, Dasaem |
| contents | Representing symbolic music with compound tokens, where each token consists of several different sub-tokens representing a distinct musical feature or attribute, offers the advantage of reducing sequence length. While previous research has validated the efficacy of compound tokens in music sequence modeling, predicting all sub-tokens simultaneously can lead to suboptimal results as it may not fully capture the interdependencies between them. We introduce the Nested Music Transformer (NMT), an architecture tailored for decoding compound tokens autoregressively, similar to processing flattened tokens, but with low memory usage. The NMT consists of two transformers: the main decoder that models a sequence of compound tokens and the sub-decoder for modeling sub-tokens of each compound token. The experiment results showed that applying the NMT to compound tokens can enhance the performance in terms of better perplexity in processing various symbolic music datasets and discrete audio tokens from the MAESTRO dataset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_01180 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Nested Music Transformer: Sequentially Decoding Compound Tokens in Symbolic Music and Audio Generation Yoo, HaeJun Dong, Hao-Wen Jung, Jongmin Jeong, Dasaem Sound Information Retrieval Machine Learning Audio and Speech Processing Representing symbolic music with compound tokens, where each token consists of several different sub-tokens representing a distinct musical feature or attribute, offers the advantage of reducing sequence length. While previous research has validated the efficacy of compound tokens in music sequence modeling, predicting all sub-tokens simultaneously can lead to suboptimal results as it may not fully capture the interdependencies between them. We introduce the Nested Music Transformer (NMT), an architecture tailored for decoding compound tokens autoregressively, similar to processing flattened tokens, but with low memory usage. The NMT consists of two transformers: the main decoder that models a sequence of compound tokens and the sub-decoder for modeling sub-tokens of each compound token. The experiment results showed that applying the NMT to compound tokens can enhance the performance in terms of better perplexity in processing various symbolic music datasets and discrete audio tokens from the MAESTRO dataset. |
| title | Nested Music Transformer: Sequentially Decoding Compound Tokens in Symbolic Music and Audio Generation |
| topic | Sound Information Retrieval Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2408.01180 |