MAEB: Massive Audio Embedding Benchmark
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914336134397952 |
|---|---|
| author | Assadi, Adnan El Chung, Isaac Xiao, Chenghao Solomatin, Roman Jha, Animesh Chand, Rahul Singh, Silky Wang, Kaitlyn Khan, Ali Sartaz Nasser, Marc Moussa Fong, Sufen He, Pengfei Xiao, Alan Munot, Ayush Sunil Shrivastava, Aditya Gazizov, Artem Muennighoff, Niklas Enevoldsen, Kenneth |
| author_facet | Assadi, Adnan El Chung, Isaac Xiao, Chenghao Solomatin, Roman Jha, Animesh Chand, Rahul Singh, Silky Wang, Kaitlyn Khan, Ali Sartaz Nasser, Marc Moussa Fong, Sufen He, Pengfei Xiao, Alan Munot, Ayush Sunil Shrivastava, Aditya Gazizov, Artem Muennighoff, Niklas Enevoldsen, Kenneth |
| contents | We introduce the Massive Audio Embedding Benchmark (MAEB), a large-scale benchmark covering 30 tasks across speech, music, environmental sounds, and cross-modal audio-text reasoning in 100+ languages. We evaluate 50+ models and find that no single model dominates across all tasks: contrastive audio-text models excel at environmental sound classification (e.g., ESC50) but score near random on multilingual speech tasks (e.g., SIB-FLEURS), while speech-pretrained models show the opposite pattern. Clustering remains challenging for all models, with even the best-performing model achieving only modest results. We observe that models excelling on acoustic understanding often perform poorly on linguistic tasks, and vice versa. We also show that the performance of audio encoders on MAEB correlates highly with their performance when used in audio large language models. MAEB is derived from MAEB+, a collection of 98 tasks. MAEB is designed to maintain task diversity while reducing evaluation cost, and it integrates into the MTEB ecosystem for unified evaluation across text, image, and audio modalities. We release MAEB and all 98 tasks along with code and a leaderboard at https://github.com/embeddings-benchmark/mteb. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_16008 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MAEB: Massive Audio Embedding Benchmark Assadi, Adnan El Chung, Isaac Xiao, Chenghao Solomatin, Roman Jha, Animesh Chand, Rahul Singh, Silky Wang, Kaitlyn Khan, Ali Sartaz Nasser, Marc Moussa Fong, Sufen He, Pengfei Xiao, Alan Munot, Ayush Sunil Shrivastava, Aditya Gazizov, Artem Muennighoff, Niklas Enevoldsen, Kenneth Sound Artificial Intelligence Computation and Language Machine Learning We introduce the Massive Audio Embedding Benchmark (MAEB), a large-scale benchmark covering 30 tasks across speech, music, environmental sounds, and cross-modal audio-text reasoning in 100+ languages. We evaluate 50+ models and find that no single model dominates across all tasks: contrastive audio-text models excel at environmental sound classification (e.g., ESC50) but score near random on multilingual speech tasks (e.g., SIB-FLEURS), while speech-pretrained models show the opposite pattern. Clustering remains challenging for all models, with even the best-performing model achieving only modest results. We observe that models excelling on acoustic understanding often perform poorly on linguistic tasks, and vice versa. We also show that the performance of audio encoders on MAEB correlates highly with their performance when used in audio large language models. MAEB is derived from MAEB+, a collection of 98 tasks. MAEB is designed to maintain task diversity while reducing evaluation cost, and it integrates into the MTEB ecosystem for unified evaluation across text, image, and audio modalities. We release MAEB and all 98 tasks along with code and a leaderboard at https://github.com/embeddings-benchmark/mteb. |
| title | MAEB: Massive Audio Embedding Benchmark |
| topic | Sound Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2602.16008 |