IncompeBench: A Permissively Licensed, Fine-Grained Benchmark for Music Information Retrieval
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911443013599232 |
|---|---|
| author | Clavié, Benjamin Shakir, Atoof Turner, Jonah Lee, Sean Shakir, Aamir Kato, Makoto P. |
| author_facet | Clavié, Benjamin Shakir, Atoof Turner, Jonah Lee, Sean Shakir, Aamir Kato, Makoto P. |
| contents | Multimodal Information Retrieval has made significant progress in recent years, leveraging the increasingly strong multimodal abilities of deep pre-trained models to represent information across modalities. Music Information Retrieval (MIR), in particular, has considerably increased in quality, with neural representations of music even making its way into everyday life products. However, there is a lack of high-quality benchmarks for evaluating music retrieval performance. To address this issue, we introduce \textbf{IncompeBench}, a carefully annotated benchmark comprising $1,574$ permissively licensed, high-quality music snippets, $500$ diverse queries, and over $125,000$ individual relevance judgements. These annotations were created through the use of a multi-stage pipeline, resulting in high agreement between human annotators and the generated data. The resulting datasets are publicly available at https://huggingface.co/datasets/mixedbread-ai/incompebench-strict and https://huggingface.co/datasets/mixedbread-ai/incompebench-lenient with the prompts available at https://github.com/mixedbread-ai/incompebench-programs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_11941 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | IncompeBench: A Permissively Licensed, Fine-Grained Benchmark for Music Information Retrieval Clavié, Benjamin Shakir, Atoof Turner, Jonah Lee, Sean Shakir, Aamir Kato, Makoto P. Information Retrieval Artificial Intelligence Multimodal Information Retrieval has made significant progress in recent years, leveraging the increasingly strong multimodal abilities of deep pre-trained models to represent information across modalities. Music Information Retrieval (MIR), in particular, has considerably increased in quality, with neural representations of music even making its way into everyday life products. However, there is a lack of high-quality benchmarks for evaluating music retrieval performance. To address this issue, we introduce \textbf{IncompeBench}, a carefully annotated benchmark comprising $1,574$ permissively licensed, high-quality music snippets, $500$ diverse queries, and over $125,000$ individual relevance judgements. These annotations were created through the use of a multi-stage pipeline, resulting in high agreement between human annotators and the generated data. The resulting datasets are publicly available at https://huggingface.co/datasets/mixedbread-ai/incompebench-strict and https://huggingface.co/datasets/mixedbread-ai/incompebench-lenient with the prompts available at https://github.com/mixedbread-ai/incompebench-programs. |
| title | IncompeBench: A Permissively Licensed, Fine-Grained Benchmark for Music Information Retrieval |
| topic | Information Retrieval Artificial Intelligence |
| url | https://arxiv.org/abs/2602.11941 |