Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.01299 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912683691868160 |
|---|---|
| author | Wang, Siyin Jin, Zengrui Tang, Changli Li, Qiujia Li, Bo Chen, Chen Hu, Yuchen Yu, Wenyi Li, Yixuan Zhuang, Jimin Yang, Yudong Wang, Mingqiu Han, Michael Ding, Yifan Bai, Junwen Ouyang, Tom Chang, Shuo-yiin Chen, Xianzhao Tian, Xiaohai Zhang, Jun Lu, Lu Sun, Guangzhi Chen, Zhehuai Wu, Ji Zhou, Bowen Wang, Yuxuan Sainath, Tara Wu, Yonghui Zhang, Chao |
| author_facet | Wang, Siyin Jin, Zengrui Tang, Changli Li, Qiujia Li, Bo Chen, Chen Hu, Yuchen Yu, Wenyi Li, Yixuan Zhuang, Jimin Yang, Yudong Wang, Mingqiu Han, Michael Ding, Yifan Bai, Junwen Ouyang, Tom Chang, Shuo-yiin Chen, Xianzhao Tian, Xiaohai Zhang, Jun Lu, Lu Sun, Guangzhi Chen, Zhehuai Wu, Ji Zhou, Bowen Wang, Yuxuan Sainath, Tara Wu, Yonghui Zhang, Chao |
| contents | In the era of large language models (LLMs) and artificial general intelligence (AGI), computer audition must evolve beyond traditional paradigms to fully leverage the capabilities of foundation models, towards more comprehensive understanding, more natural generation and more human-like interaction. Audio, as a modality rich in semantic, emotional, and contextual cues, plays a vital role in achieving naturalistic and embodied machine intelligence. This survey provides a comprehensive review of recent progress in integrating audio into LLMs, with a focus on four key areas: audio comprehension, audio generation, speech-based interaction, and audio-visual understanding. We analyze how LLMs are reshaping audio perception and reasoning, enabling systems to understand sound at a deeper semantic level, generate expressive audio outputs, and engage in human-like spoken interaction. Furthermore, we explore how the fusion of audio and visual modalities enhances situational awareness and cross-modal reasoning, pushing the boundaries of multimodal intelligence. This survey not only synthesizes existing research but also identifies critical challenges and future directions for building audio-native AGI systems capable of perceiving, understanding, and interacting through sound as naturally as humans do. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_01299 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards General Auditory Intelligence: Large Multimodal Models for Machine Listening and Speaking Wang, Siyin Jin, Zengrui Tang, Changli Li, Qiujia Li, Bo Chen, Chen Hu, Yuchen Yu, Wenyi Li, Yixuan Zhuang, Jimin Yang, Yudong Wang, Mingqiu Han, Michael Ding, Yifan Bai, Junwen Ouyang, Tom Chang, Shuo-yiin Chen, Xianzhao Tian, Xiaohai Zhang, Jun Lu, Lu Sun, Guangzhi Chen, Zhehuai Wu, Ji Zhou, Bowen Wang, Yuxuan Sainath, Tara Wu, Yonghui Zhang, Chao Audio and Speech Processing In the era of large language models (LLMs) and artificial general intelligence (AGI), computer audition must evolve beyond traditional paradigms to fully leverage the capabilities of foundation models, towards more comprehensive understanding, more natural generation and more human-like interaction. Audio, as a modality rich in semantic, emotional, and contextual cues, plays a vital role in achieving naturalistic and embodied machine intelligence. This survey provides a comprehensive review of recent progress in integrating audio into LLMs, with a focus on four key areas: audio comprehension, audio generation, speech-based interaction, and audio-visual understanding. We analyze how LLMs are reshaping audio perception and reasoning, enabling systems to understand sound at a deeper semantic level, generate expressive audio outputs, and engage in human-like spoken interaction. Furthermore, we explore how the fusion of audio and visual modalities enhances situational awareness and cross-modal reasoning, pushing the boundaries of multimodal intelligence. This survey not only synthesizes existing research but also identifies critical challenges and future directions for building audio-native AGI systems capable of perceiving, understanding, and interacting through sound as naturally as humans do. |
| title | Towards General Auditory Intelligence: Large Multimodal Models for Machine Listening and Speaking |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2511.01299 |