Towards Reliable Large Audio Language Model
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918034381209600 |
|---|---|
| author | Ma, Ziyang Li, Xiquan Song, Yakun Chen, Wenxi Du, Chenpeng Wu, Jian Chen, Yuanzhe Chen, Zhuo Wang, Yuping Wang, Yuxuan Chen, Xie |
| author_facet | Ma, Ziyang Li, Xiquan Song, Yakun Chen, Wenxi Du, Chenpeng Wu, Jian Chen, Yuanzhe Chen, Zhuo Wang, Yuping Wang, Yuxuan Chen, Xie |
| contents | Recent advancements in large audio language models (LALMs) have demonstrated impressive results and promising prospects in universal understanding and reasoning across speech, music, and general sound. However, these models still lack the ability to recognize their knowledge boundaries and refuse to answer questions they don't know proactively. While there have been successful attempts to enhance the reliability of LLMs, reliable LALMs remain largely unexplored. In this paper, we systematically investigate various approaches towards reliable LALMs, including training-free methods such as multi-modal chain-of-thought (MCoT), and training-based methods such as supervised fine-tuning (SFT). Besides, we identify the limitations of previous evaluation metrics and propose a new metric, the Reliability Gain Index (RGI), to assess the effectiveness of different reliable methods. Our findings suggest that both training-free and training-based methods enhance the reliability of LALMs to different extents. Moreover, we find that awareness of reliability is a "meta ability", which can be transferred across different audio modalities, although significant structural and content differences exist among sound, music, and speech. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_19294 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Reliable Large Audio Language Model Ma, Ziyang Li, Xiquan Song, Yakun Chen, Wenxi Du, Chenpeng Wu, Jian Chen, Yuanzhe Chen, Zhuo Wang, Yuping Wang, Yuxuan Chen, Xie Sound Computation and Language Human-Computer Interaction Multimedia Audio and Speech Processing Recent advancements in large audio language models (LALMs) have demonstrated impressive results and promising prospects in universal understanding and reasoning across speech, music, and general sound. However, these models still lack the ability to recognize their knowledge boundaries and refuse to answer questions they don't know proactively. While there have been successful attempts to enhance the reliability of LLMs, reliable LALMs remain largely unexplored. In this paper, we systematically investigate various approaches towards reliable LALMs, including training-free methods such as multi-modal chain-of-thought (MCoT), and training-based methods such as supervised fine-tuning (SFT). Besides, we identify the limitations of previous evaluation metrics and propose a new metric, the Reliability Gain Index (RGI), to assess the effectiveness of different reliable methods. Our findings suggest that both training-free and training-based methods enhance the reliability of LALMs to different extents. Moreover, we find that awareness of reliability is a "meta ability", which can be transferred across different audio modalities, although significant structural and content differences exist among sound, music, and speech. |
| title | Towards Reliable Large Audio Language Model |
| topic | Sound Computation and Language Human-Computer Interaction Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2505.19294 |