MusicSem: A Semantically Rich Language--Audio Dataset of Natural Music Descriptions
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917283731865600 |
|---|---|
| author | Salganik, Rebecca Tu, Teng Chen, Fei-Yueh Liu, Xiaohao Lu, Keifeng Luvisia, Ethan Duan, Zhiyao Salha-Galvan, Guillaume Kahng, Anson Ma, Yunshan Kang, Jian |
| author_facet | Salganik, Rebecca Tu, Teng Chen, Fei-Yueh Liu, Xiaohao Lu, Keifeng Luvisia, Ethan Duan, Zhiyao Salha-Galvan, Guillaume Kahng, Anson Ma, Yunshan Kang, Jian |
| contents | Music representation learning is central to music information retrieval and generation. While recent advances in multimodal learning have improved alignment between text and audio for tasks such as cross-modal music retrieval, text-to-music generation, and music-to-text generation, existing models often struggle to capture users' expressed intent in natural language descriptions of music. This observation suggests that the datasets used to train and evaluate these models do not fully reflect the broader and more natural forms of human discourse through which music is described. In this paper, we introduce MusicSem, a dataset of 32,493 language-audio pairs derived from organic music-related discussions on the social media platform Reddit. Compared to existing datasets, MusicSem captures a broader spectrum of musical semantics, reflecting how listeners naturally describe music in nuanced and human-centered ways. To structure these expressions, we propose a taxonomy of five semantic categories: descriptive, atmospheric, situational, metadata-related, and contextual. In addition to the construction, analysis, and release of MusicSem, we use the dataset to evaluate a wide range of multimodal models for retrieval and generation, highlighting the importance of modeling fine-grained semantics. Overall, MusicSem serves as a novel semantics-aware resource to support future research on human-aligned multimodal music representation learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_17769 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MusicSem: A Semantically Rich Language--Audio Dataset of Natural Music Descriptions Salganik, Rebecca Tu, Teng Chen, Fei-Yueh Liu, Xiaohao Lu, Keifeng Luvisia, Ethan Duan, Zhiyao Salha-Galvan, Guillaume Kahng, Anson Ma, Yunshan Kang, Jian Multimedia Sound Audio and Speech Processing Music representation learning is central to music information retrieval and generation. While recent advances in multimodal learning have improved alignment between text and audio for tasks such as cross-modal music retrieval, text-to-music generation, and music-to-text generation, existing models often struggle to capture users' expressed intent in natural language descriptions of music. This observation suggests that the datasets used to train and evaluate these models do not fully reflect the broader and more natural forms of human discourse through which music is described. In this paper, we introduce MusicSem, a dataset of 32,493 language-audio pairs derived from organic music-related discussions on the social media platform Reddit. Compared to existing datasets, MusicSem captures a broader spectrum of musical semantics, reflecting how listeners naturally describe music in nuanced and human-centered ways. To structure these expressions, we propose a taxonomy of five semantic categories: descriptive, atmospheric, situational, metadata-related, and contextual. In addition to the construction, analysis, and release of MusicSem, we use the dataset to evaluate a wide range of multimodal models for retrieval and generation, highlighting the importance of modeling fine-grained semantics. Overall, MusicSem serves as a novel semantics-aware resource to support future research on human-aligned multimodal music representation learning. |
| title | MusicSem: A Semantically Rich Language--Audio Dataset of Natural Music Descriptions |
| topic | Multimedia Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2602.17769 |