SAM Audio Judge: A Unified Multimodal Framework for Perceptual Evaluation of Audio Separation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914283750686720 |
|---|---|
| author | Wang, Helin Shi, Bowen Tjandra, Andros Hoffman, John Wu, Yi-Chiao Vyas, Apoorv Dehak, Najim Lee, Ann Hsu, Wei-Ning |
| author_facet | Wang, Helin Shi, Bowen Tjandra, Andros Hoffman, John Wu, Yi-Chiao Vyas, Apoorv Dehak, Najim Lee, Ann Hsu, Wei-Ning |
| contents | The performance evaluation remains a complex challenge in audio separation, and existing evaluation metrics are often misaligned with human perception, course-grained, relying on ground truth signals. On the other hand, subjective listening tests remain the gold standard for real-world evaluation, but they are expensive, time-consuming, and difficult to scale. This paper addresses the growing need for automated systems capable of evaluating audio separation without human intervention. The proposed evaluation metric, SAM Audio Judge (SAJ), is a multimodal fine-grained reference-free objective metric, which shows highly alignment with human perceptions. SAJ supports three audio domains (speech, music and general sound events) and three prompt inputs (text, visual and span), covering four different dimensions of evaluation (recall, percision, faithfulness, and overall). SAM Audio Judge also shows potential applications in data filtering, pseudo-labeling large datasets and reranking in audio separation models. We release our code and pre-trained models at: https://github.com/facebookresearch/sam-audio. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_19702 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SAM Audio Judge: A Unified Multimodal Framework for Perceptual Evaluation of Audio Separation Wang, Helin Shi, Bowen Tjandra, Andros Hoffman, John Wu, Yi-Chiao Vyas, Apoorv Dehak, Najim Lee, Ann Hsu, Wei-Ning Audio and Speech Processing Artificial Intelligence The performance evaluation remains a complex challenge in audio separation, and existing evaluation metrics are often misaligned with human perception, course-grained, relying on ground truth signals. On the other hand, subjective listening tests remain the gold standard for real-world evaluation, but they are expensive, time-consuming, and difficult to scale. This paper addresses the growing need for automated systems capable of evaluating audio separation without human intervention. The proposed evaluation metric, SAM Audio Judge (SAJ), is a multimodal fine-grained reference-free objective metric, which shows highly alignment with human perceptions. SAJ supports three audio domains (speech, music and general sound events) and three prompt inputs (text, visual and span), covering four different dimensions of evaluation (recall, percision, faithfulness, and overall). SAM Audio Judge also shows potential applications in data filtering, pseudo-labeling large datasets and reranking in audio separation models. We release our code and pre-trained models at: https://github.com/facebookresearch/sam-audio. |
| title | SAM Audio Judge: A Unified Multimodal Framework for Perceptual Evaluation of Audio Separation |
| topic | Audio and Speech Processing Artificial Intelligence |
| url | https://arxiv.org/abs/2601.19702 |