Cover Image

End-to-end Listen, Look, Speak and Act

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Siyin, Yu, Wenyi, Chen, Xianzhao, Tian, Xiaohai, Zhang, Jun, Lu, Lu, Zhang, Chao
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Robotics Audio and Speech Processing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions
by: Wang, Siyin, et al.
Published: (2025)

SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation
by: Yu, Wenyi, et al.
Published: (2024)

Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation
by: Wang, Siyin, et al.
Published: (2024)

Bayesian Example Selection Improves In-Context Learning for Speech, Text, and Visual Modalities
by: Wang, Siyin, et al.
Published: (2024)

Anatomy of the Modality Gap: Dissecting the Internal States of End-to-End Speech LLMs
by: Hsu, Ming-Hao, et al.
Published: (2026)

Look, Listen and Recognise: Character-Aware Audio-Visual Subtitling
by: Korbar, Bruno, et al.
Published: (2024)

Robot Confirmation Generation and Action Planning Using Long-context Q-Former Integrated with Multimodal LLM
by: Hori, Chiori, et al.
Published: (2025)

SALMONN: Towards Generic Hearing Abilities for Large Language Models
by: Tang, Changli, et al.
Published: (2023)

Listening without Looking: Modality Bias in Audio-Visual Captioning
by: Ishikawa, Yuchi, et al.
Published: (2025)

ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of Video
by: Cai, Kevin, et al.
Published: (2024)

SynesLM: A Unified Approach for Audio-visual Speech Recognition and Translation via Language Model and Synthetic Data
by: Lu, Yichen, et al.
Published: (2024)

TCDiff++: An End-to-end Trajectory-Controllable Diffusion Model for Harmonious Music-Driven Group Choreography
by: Dai, Yuqin, et al.
Published: (2025)

Towards General Auditory Intelligence: Large Multimodal Models for Machine Listening and Speaking
by: Wang, Siyin, et al.
Published: (2025)

CustomListener: Text-guided Responsive Interaction for User-friendly Listening Head Generation
by: Liu, Xi, et al.
Published: (2024)

VAPO: End-to-end Slide-Enhanced Speech Recognition with Omni-modal Large Language Models
by: Hu, Rui, et al.
Published: (2025)

Robust Audiovisual Speech Recognition Models with Mixture-of-Experts
by: Wu, Yihan, et al.
Published: (2024)

ActiveRIR: Active Audio-Visual Exploration for Acoustic Environment Modeling
by: Somayazulu, Arjun, et al.
Published: (2024)

End-to-end Audio Deepfake Detection from RAW Waveforms: a RawNet-Based Approach with Cross-Dataset Evaluation
by: Di Pierno, Andrea, et al.
Published: (2025)

How Should LLMs Listen While Speaking? A Study of User-Stream Routing in Full-Duplex Spoken Dialogue
by: Lu, Hui, et al.
Published: (2026)

Data Augmentation for End-to-end Code-switching Speech Recognition
by: Du, Chenpeng, et al.
Published: (2020)

A-JEPA: Joint-Embedding Predictive Architecture Can Listen
by: Fei, Zhengcong, et al.
Published: (2023)

Advancing Automated Spatio-Semantic Analysis in Picture Description Using Language Models
by: Ng, Si-Ioi, et al.
Published: (2025)

Improved Visually Prompted Keyword Localisation in Real Low-Resource Settings
by: Nortje, Leanne, et al.
Published: (2024)

Inter-Diffusion Generation Model of Speakers and Listeners for Effective Communication
by: Huang, Jinhe, et al.
Published: (2025)

DeepAudio-V1:Towards Multi-Modal Multi-Stage End-to-End Video to Speech and Audio Generation
by: Zhang, Haomin, et al.
Published: (2025)

Seed LiveInterpret 2.0: End-to-end Simultaneous Speech-to-speech Translation with Your Voice
by: Cheng, Shanbo, et al.
Published: (2025)

SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words
by: Ao, Junyi, et al.
Published: (2024)

Solla: Towards a Speech-Oriented LLM That Hears Acoustic Context
by: Ao, Junyi, et al.
Published: (2025)

EchoTrack: Auditory Referring Multi-Object Tracking for Autonomous Driving
by: Lin, Jiacheng, et al.
Published: (2024)

Can Whisper perform speech-based in-context learning?
by: Wang, Siyin, et al.
Published: (2023)

Qwen2.5-Omni Technical Report
by: Xu, Jin, et al.
Published: (2025)

Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Models
by: Lin, Yi-Cheng, et al.
Published: (2024)

Sheet Music Transformer: End-To-End Optical Music Recognition Beyond Monophonic Transcription
by: Ríos-Vila, Antonio, et al.
Published: (2024)

EmoTalker: Emotionally Editable Talking Face Generation via Diffusion Model
by: Zhang, Bingyuan, et al.
Published: (2024)

NaturalL2S: End-to-End High-quality Multispeaker Lip-to-Speech Synthesis with Differential Digital Signal Processing
by: Liang, Yifan, et al.
Published: (2025)

Aligned Better, Listen Better for Audio-Visual Large Language Models
by: Guo, Yuxin, et al.
Published: (2025)

SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generation
by: Zhang, Youliang, et al.
Published: (2025)

Measuring Sound Symbolism in Audio-visual Models
by: Tseng, Wei-Cheng, et al.
Published: (2024)

Better Spanish Emotion Recognition In-the-wild: Bringing Attention to Deep Spectrum Voice Analysis
by: Ortega-Beltrán, Elena, et al.
Published: (2024)

Noise-Robust AV-ASR Using Visual Features Both in the Whisper Encoder and Decoder
by: Li, Zhengyang, et al.
Published: (2026)