Do Foundational Audio Encoders Understand Music Structure?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908795318304768 |
|---|---|
| author | Toyama, Keisuke Zhong, Zhi Takahashi, Akira Takahashi, Shusuke Mitsufuji, Yuki |
| author_facet | Toyama, Keisuke Zhong, Zhi Takahashi, Akira Takahashi, Shusuke Mitsufuji, Yuki |
| contents | In music information retrieval (MIR) research, the use of pretrained foundational audio encoders (FAEs) has recently become a trend. FAEs pretrained on large amounts of music and audio data have been shown to improve performance on MIR tasks such as music tagging and automatic music transcription. However, their use for music structure analysis (MSA) remains underexplored: only a small subset of FAEs has been examined for MSA, and the impact of factors such as learning methods, training data, and model context length on MSA performance remains unclear. In this study, we conduct comprehensive experiments on 11 types of FAEs to investigate how these factors affect MSA performance. Our results demonstrate that FAEs using self-supervised learning with masked language modeling on music data are particularly effective for MSA. These findings pave the way for future research in FAE and MSA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_17209 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Do Foundational Audio Encoders Understand Music Structure? Toyama, Keisuke Zhong, Zhi Takahashi, Akira Takahashi, Shusuke Mitsufuji, Yuki Sound Machine Learning Audio and Speech Processing In music information retrieval (MIR) research, the use of pretrained foundational audio encoders (FAEs) has recently become a trend. FAEs pretrained on large amounts of music and audio data have been shown to improve performance on MIR tasks such as music tagging and automatic music transcription. However, their use for music structure analysis (MSA) remains underexplored: only a small subset of FAEs has been examined for MSA, and the impact of factors such as learning methods, training data, and model context length on MSA performance remains unclear. In this study, we conduct comprehensive experiments on 11 types of FAEs to investigate how these factors affect MSA performance. Our results demonstrate that FAEs using self-supervised learning with masked language modeling on music data are particularly effective for MSA. These findings pave the way for future research in FAE and MSA. |
| title | Do Foundational Audio Encoders Understand Music Structure? |
| topic | Sound Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2512.17209 |