Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.11098 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910016043220992 |
|---|---|
| author | Hu, Jiliang Wang, Wenfu Li, Zuchao Li, Chenxing Zhao, Yiyang Li, Hanzhao Zhang, Liqiang Yu, Meng Yu, Dong |
| author_facet | Hu, Jiliang Wang, Wenfu Li, Zuchao Li, Chenxing Zhao, Yiyang Li, Hanzhao Zhang, Liqiang Yu, Meng Yu, Dong |
| contents | Recent advances in large audio language models (LALMs) have greatly enhanced multimodal conversational systems. However, existing benchmarks remain limited -- they are mainly English-centric, rely on synthetic speech, and lack comprehensive, discriminative evaluation across multiple dimensions. To address these gaps, we present Voice Chat Bot Bench (VCB Bench) -- a high-quality Chinese benchmark built entirely on real human speech. VCB Bench evaluates LALMs from three complementary perspectives: instruction following (including speech-level control beyond text commands), knowledge understanding (general knowledge, reasoning, and daily dialogue), and robustness (stability under perturbations in content, environment, and speaker traits). Experiments on representative LALMs reveal notable performance gaps and highlight future directions for improvement. VCB Bench provides a reproducible and fine-grained evaluation framework, offering standardized methodology and practical insights for advancing Chinese voice conversational models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_11098 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VCB Bench: An Evaluation Benchmark for Audio-Grounded Large Language Model Conversational Agents Hu, Jiliang Wang, Wenfu Li, Zuchao Li, Chenxing Zhao, Yiyang Li, Hanzhao Zhang, Liqiang Yu, Meng Yu, Dong Sound Computation and Language Recent advances in large audio language models (LALMs) have greatly enhanced multimodal conversational systems. However, existing benchmarks remain limited -- they are mainly English-centric, rely on synthetic speech, and lack comprehensive, discriminative evaluation across multiple dimensions. To address these gaps, we present Voice Chat Bot Bench (VCB Bench) -- a high-quality Chinese benchmark built entirely on real human speech. VCB Bench evaluates LALMs from three complementary perspectives: instruction following (including speech-level control beyond text commands), knowledge understanding (general knowledge, reasoning, and daily dialogue), and robustness (stability under perturbations in content, environment, and speaker traits). Experiments on representative LALMs reveal notable performance gaps and highlight future directions for improvement. VCB Bench provides a reproducible and fine-grained evaluation framework, offering standardized methodology and practical insights for advancing Chinese voice conversational models. |
| title | VCB Bench: An Evaluation Benchmark for Audio-Grounded Large Language Model Conversational Agents |
| topic | Sound Computation and Language |
| url | https://arxiv.org/abs/2510.11098 |