Singing Timbre Popularity Assessment Based on Multimodal Large Foundation Model
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917132094144512 |
|---|---|
| author | Wang, Zihao Yuan, Ruibin Geng, Ziqi Li, Hengjia Qu, Xingwei Li, Xinyi Chen, Songye Fu, Haoying Dannenberg, Roger B. Zhang, Kejun |
| author_facet | Wang, Zihao Yuan, Ruibin Geng, Ziqi Li, Hengjia Qu, Xingwei Li, Xinyi Chen, Songye Fu, Haoying Dannenberg, Roger B. Zhang, Kejun |
| contents | Automated singing assessment is crucial for education and entertainment. However, existing systems face two fundamental limitations: reliance on reference tracks, which stifles creative expression, and the simplification of complex performances into non-diagnostic scores based solely on pitch and rhythm. We advocate for a shift from discriminative to descriptive evaluation, creating a complete ecosystem for reference-free, multi-dimensional assessment. First, we introduce Sing-MD, a large-scale dataset annotated by experts across four dimensions: breath control, timbre quality, emotional expression, and vocal technique. Our analysis reveals significant annotation inconsistencies among experts, challenging the validity of traditional accuracy-based metrics. Second, addressing the memory limitations of Multimodal Large Language Models (MLLMs) in analyzing full-length songs, we propose VocalVerse. This efficient hybrid architecture leverages a lightweight acoustic encoder to model global performance features and long-term dependencies. Third, to address automated metric shortcomings, we establish the H-TPR (Human-in-the-loop Tiered Perceptual Ranking) benchmark, which evaluates a model's ability to generate perceptually valid rankings rather than predicting noisy ground-truth scores. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_06999 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Singing Timbre Popularity Assessment Based on Multimodal Large Foundation Model Wang, Zihao Yuan, Ruibin Geng, Ziqi Li, Hengjia Qu, Xingwei Li, Xinyi Chen, Songye Fu, Haoying Dannenberg, Roger B. Zhang, Kejun Sound Artificial Intelligence H.5.5; I.2.7 Automated singing assessment is crucial for education and entertainment. However, existing systems face two fundamental limitations: reliance on reference tracks, which stifles creative expression, and the simplification of complex performances into non-diagnostic scores based solely on pitch and rhythm. We advocate for a shift from discriminative to descriptive evaluation, creating a complete ecosystem for reference-free, multi-dimensional assessment. First, we introduce Sing-MD, a large-scale dataset annotated by experts across four dimensions: breath control, timbre quality, emotional expression, and vocal technique. Our analysis reveals significant annotation inconsistencies among experts, challenging the validity of traditional accuracy-based metrics. Second, addressing the memory limitations of Multimodal Large Language Models (MLLMs) in analyzing full-length songs, we propose VocalVerse. This efficient hybrid architecture leverages a lightweight acoustic encoder to model global performance features and long-term dependencies. Third, to address automated metric shortcomings, we establish the H-TPR (Human-in-the-loop Tiered Perceptual Ranking) benchmark, which evaluates a model's ability to generate perceptually valid rankings rather than predicting noisy ground-truth scores. |
| title | Singing Timbre Popularity Assessment Based on Multimodal Large Foundation Model |
| topic | Sound Artificial Intelligence H.5.5; I.2.7 |
| url | https://arxiv.org/abs/2512.06999 |