Can Impressions of Music be Extracted from Thumbnail Images?
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913636605231104 |
|---|---|
| author | Harada, Takashi Motomitsu, Takehiro Hayashi, Katsuhiko Sakai, Yusuke Kamigaito, Hidetaka |
| author_facet | Harada, Takashi Motomitsu, Takehiro Hayashi, Katsuhiko Sakai, Yusuke Kamigaito, Hidetaka |
| contents | In recent years, there has been a notable increase in research on machine learning models for music retrieval and generation systems that are capable of taking natural language sentences as inputs. However, there is a scarcity of large-scale publicly available datasets, consisting of music data and their corresponding natural language descriptions known as music captions. In particular, non-musical information such as suitable situations for listening to a track and the emotions elicited upon listening is crucial for describing music. This type of information is underrepresented in existing music caption datasets due to the challenges associated with extracting it directly from music data. To address this issue, we propose a method for generating music caption data that incorporates non-musical aspects inferred from music thumbnail images, and validated the effectiveness of our approach through human evaluations. Additionally, we created a dataset with approximately 360,000 captions containing non-musical aspects. Leveraging this dataset, we trained a music retrieval model and demonstrated its effectiveness in music retrieval tasks through evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_02511 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Can Impressions of Music be Extracted from Thumbnail Images? Harada, Takashi Motomitsu, Takehiro Hayashi, Katsuhiko Sakai, Yusuke Kamigaito, Hidetaka Computation and Language Computer Vision and Pattern Recognition Information Retrieval Sound Audio and Speech Processing In recent years, there has been a notable increase in research on machine learning models for music retrieval and generation systems that are capable of taking natural language sentences as inputs. However, there is a scarcity of large-scale publicly available datasets, consisting of music data and their corresponding natural language descriptions known as music captions. In particular, non-musical information such as suitable situations for listening to a track and the emotions elicited upon listening is crucial for describing music. This type of information is underrepresented in existing music caption datasets due to the challenges associated with extracting it directly from music data. To address this issue, we propose a method for generating music caption data that incorporates non-musical aspects inferred from music thumbnail images, and validated the effectiveness of our approach through human evaluations. Additionally, we created a dataset with approximately 360,000 captions containing non-musical aspects. Leveraging this dataset, we trained a music retrieval model and demonstrated its effectiveness in music retrieval tasks through evaluation. |
| title | Can Impressions of Music be Extracted from Thumbnail Images? |
| topic | Computation and Language Computer Vision and Pattern Recognition Information Retrieval Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2501.02511 |