Cover Image

Towards General Auditory Intelligence: Large Multimodal Models for Machine Listening and Speaking

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Siyin, Jin, Zengrui, Tang, Changli, Li, Qiujia, Li, Bo, Chen, Chen, Hu, Yuchen, Yu, Wenyi, Li, Yixuan, Zhuang, Jimin, Yang, Yudong, Wang, Mingqiu, Han, Michael, Ding, Yifan, Bai, Junwen, Ouyang, Tom, Chang, Shuo-yiin, Chen, Xianzhao, Tian, Xiaohai, Zhang, Jun, Lu, Lu, Sun, Guangzhi, Chen, Zhehuai, Wu, Ji, Zhou, Bowen, Wang, Yuxuan, Sainath, Tara, Wu, Yonghui, Zhang, Chao
Format:	Preprint
Published:	2025
Subjects:	Audio and Speech Processing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

End-to-end Listen, Look, Speak and Act
by: Wang, Siyin, et al.
Published: (2025)

Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation
by: Wang, Siyin, et al.
Published: (2024)

SALMONN-omni: A Standalone Speech LLM without Codec Injection for Full-duplex Conversation
by: Yu, Wenyi, et al.
Published: (2025)

SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation
by: Yu, Wenyi, et al.
Published: (2024)

Efficient Adapter Finetuning for Tail Languages in Streaming Multilingual ASR
by: Bai, Junwen, et al.
Published: (2024)

QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions
by: Wang, Siyin, et al.
Published: (2025)

SALMONN: Towards Generic Hearing Abilities for Large Language Models
by: Tang, Changli, et al.
Published: (2023)

video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models
by: Sun, Guangzhi, et al.
Published: (2024)

OCR-Enhanced Multimodal ASR Can Read While Listening
by: Chen, Junli, et al.
Published: (2026)

Can Large Language Models Understand Spatial Audio?
by: Tang, Changli, et al.
Published: (2024)

Speech-Audio Compositional Attacks on Multimodal LLMs and Their Mitigation with SALMONN-Guard
by: Yang, Yudong, et al.
Published: (2025)

Handling Ambiguity in Emotion: From Out-of-Domain Detection to Distribution Estimation
by: Wu, Wen, et al.
Published: (2024)

The Effect of Alignment on the Listening and Speaking Continuation Task
by: Yang Zhang, et al.
Published: (2024)

Language Model Can Listen While Speaking
by: Ma, Ziyang, et al.
Published: (2024)

video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Models
by: Tang, Changli, et al.
Published: (2025)

video-SALMONN-o1: Reasoning-enhanced Audio-visual Large Language Model
by: Sun, Guangzhi, et al.
Published: (2025)

Improving LLM Video Understanding with 16 Frames Per Second
by: Li, Yixuan, et al.
Published: (2025)

Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization
by: Tang, Changli, et al.
Published: (2024)

How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialogue
by: Lu, Hui, et al.
Published: (2026)

Towards Autonomous Business Intelligence via Data-to-Insight Discovery Agent
by: Wu, Dongming, et al.
Published: (2026)

ListenNet: A Lightweight Spatio-Temporal Enhancement Nested Network for Auditory Attention Detection
by: Fan, Cunhang, et al.
Published: (2025)

VoiceAssistant-Eval: Benchmarking AI Assistants across Listening, Speaking, and Viewing
by: Wang, Ke, et al.
Published: (2025)

Audio Large Language Models Can Be Descriptive Speech Quality Evaluators
by: Chen, Chen, et al.
Published: (2025)

MetaSICL: Adapting Audiroty LLM via Meta Speech In-Context Learning
by: Zheng, Haolong, et al.
Published: (2026)

Audio-centric Video Understanding Benchmark without Text Shortcut
by: Yang, Yudong, et al.
Published: (2025)

Augmenting Open-Vocabulary Dysarthric Speech Assessment with Human Perceptual Supervision
by: Jia, Kaimeng, et al.
Published: (2025)

SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words
by: Ao, Junyi, et al.
Published: (2024)

Hijacking Large Audio-Language Models via Context-Agnostic and Imperceptible Auditory Prompt Injection
by: Chen, Meng, et al.
Published: (2026)

Solla: Towards a Speech-Oriented LLM That Hears Acoustic Context
by: Ao, Junyi, et al.
Published: (2025)

Bayesian‐optimized deep learning for identifying essential genes of mitophagy and fostering therapies to combat drug resistance in human cancers
by: Wenyi Jin, et al.
Published: (2025)

DrawSpeech: Expressive Speech Synthesis Using Prosodic Sketches as Control Conditions
by: Chen, Weidong, et al.
Published: (2025)

STORM: End-to-End Referring Multi-Object Tracking in Videos
by: Lu, Zijia, et al.
Published: (2026)

SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Models
by: Yang, Chih-Kai, et al.
Published: (2025)

CoAVT: A Cognition-Inspired Unified Audio-Visual-Text Pre-Training Model for Multimodal Processing
by: Yue, Xianghu, et al.
Published: (2024)

Study of Selective Catalytic Reduction Performance of Ce‐Modified Iron‐Based SAPO‐34 Catalysts
by: Kun Li, et al.
Published: (2025)

Latent Profiles of Chinese Undergraduates’ English Speaking Anxiety and Enjoyment: Associations With Speaking Self‐Efficacy
by: Xinle Chen, et al.
Published: (2025)

Anti‐Apoptotic Efficacies of Iron Nanoparticles Green‐Formulated by Syzygium aromaticum Leaf Extract on NCI‐H661 Human Lung Adenocarcinoma Cell Line and Following the PI3K/AKT/mTOR Signaling Pathway
by: Yue Li, et al.
Published: (2025)

Bifunctional Interface Synergy for High‐Performance Wide‐Bandgap and Tandem Perovskite Solar Cells
by: Chen Jia, et al.
Published: (2026)

83‐3: Inverse Design of Liquid Crystal Phase Modulators for 2D/3D Switchable Display Based on Deep Learning
by: Qian Chen, et al.
Published: (2024)

Listen, Chat, and Remix: Text-Guided Soundscape Remixing for Enhanced Auditory Experience
by: Jiang, Xilin, et al.
Published: (2024)