Image de couverture de livre

Omni-AVSR: Towards Unified Multimodal Speech Recognition with Large Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Cappellazzo, Umberto, Liu, Xubo, Ma, Pingchuan, Petridis, Stavros, Pantic, Maja
Format:	Preprint
Publié:	2025
Sujets:	Audio and Speech Processing Computer Vision and Pattern Recognition Sound
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Dr. SHAP-AV: Decoding Relative Modality Contributions via Shapley Attribution in Audio-Visual Speech Recognition
par: Cappellazzo, Umberto, et autres
Publié: (2026)

MoME: Mixture of Matryoshka Experts for Audio-Visual Speech Recognition
par: Cappellazzo, Umberto, et autres
Publié: (2025)

Mitigating Attention Sinks and Massive Activations in Audio-Visual Speech Recognition with LLMs
par: Anand, et autres
Publié: (2025)

Large Language Models are Strong Audio-Visual Speech Recognition Learners
par: Cappellazzo, Umberto, et autres
Publié: (2024)

Adaptive Audio-Visual Speech Recognition via Matryoshka-Based Multimodal LLMs
par: Cappellazzo, Umberto, et autres
Publié: (2025)

Scaling and Enhancing LLM-based AVSR: A Sparse Mixture of Projectors Approach
par: Cappellazzo, Umberto, et autres
Publié: (2025)

Zero-AVSR: Zero-Shot Audio-Visual Speech Recognition with LLMs by Learning Language-Agnostic Speech Representations
par: Yeo, Jeong Hun, et autres
Publié: (2025)

RT-LA-VocE: Real-Time Low-SNR Audio-Visual Speech Enhancement
par: Chen, Honglie, et autres
Publié: (2024)

Full-Rank No More: Low-Rank Weight Training for Modern Speech Recognition Models
par: Fernandez-Lopez, Adriana, et autres
Publié: (2024)

Dynamic Data Pruning for Automatic Speech Recognition
par: Xiao, Qiao, et autres
Publié: (2024)

Contextual Speech Extraction: Leveraging Textual History as an Implicit Cue for Target Speech Extraction
par: Kim, Minsu, et autres
Publié: (2025)

VAPO: End-to-end Slide-Enhanced Speech Recognition with Omni-modal Large Language Models
par: Hu, Rui, et autres
Publié: (2025)

AD-AVSR: Asymmetric Dual-stream Enhancement for Robust Audio-Visual Speech Recognition
par: Xue, Junxiao, et autres
Publié: (2025)

SlideAVSR: A Dataset of Paper Explanation Videos for Audio-Visual Speech Recognition
par: Wang, Hao, et autres
Publié: (2024)

Revival with Voice: Multi-modal Controllable Text-to-Speech Synthesis
par: Kim, Minsu, et autres
Publié: (2025)

Large-scale unsupervised audio pre-training for video-to-speech synthesis
par: Kefalas, Triantafyllos, et autres
Publié: (2023)

DCIM-AVSR : Efficient Audio-Visual Speech Recognition via Dual Conformer Interaction Module
par: Wang, Xinyu, et autres
Publié: (2024)

AudioGen-Omni: A Unified Multimodal Diffusion Transformer for Video-Synchronized Audio, Speech, and Song Generation
par: Wang, Le, et autres
Publié: (2025)

Audio-visual video-to-speech synthesis with synthesized input audio
par: Kefalas, Triantafyllos, et autres
Publié: (2023)

OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup
par: Cheng, Xize, et autres
Publié: (2024)

Leveraging Large Language Models in Visual Speech Recognition: Model Scaling, Context-Aware Decoding, and Iterative Polishing
par: Liu, Zehua, et autres
Publié: (2025)

Stream-Omni: Simultaneous Multimodal Interactions with Large Language-Vision-Speech Model
par: Zhang, Shaolei, et autres
Publié: (2025)

UniCUE: Unified Recognition and Generation Framework for Chinese Cued Speech Video-to-Speech Generation
par: Wang, Jinting, et autres
Publié: (2025)

Zero-Shot Recognition of Dysarthric Speech Using Commercial Automatic Speech Recognition and Multimodal Large Language Models
par: Alsayegh, Ali, et autres
Publié: (2025)

Pseudo Labels-based Neural Speech Enhancement for the AVSR Task in the MISP-Meeting Challenge
par: Luo, Longjie, et autres
Publié: (2025)

DualSpeechLM: Towards Unified Speech Understanding and Generation via Dual Speech Token Modeling with Large Language Models
par: Wang, Yuanyuan, et autres
Publié: (2025)

MLCA-AVSR: Multi-Layer Cross Attention Fusion based Audio-Visual Speech Recognition
par: Wang, He, et autres
Publié: (2024)

S2ST-Omni: Hierarchical Language-Aware SpeechLLM Adaptation for Multilingual Speech-to-Speech Translation
par: Pan, Yu, et autres
Publié: (2025)

Exploring Local Interpretable Model-Agnostic Explanations for Speech Emotion Recognition with Distribution-Shift
par: Hjuler, Maja J., et autres
Publié: (2025)

Enhancing CTC-Based Visual Speech Recognition
par: Laux, Hendrik, et autres
Publié: (2024)

Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision
par: Liu, Che, et autres
Publié: (2025)

MMS-LLaMA: Efficient LLM-based Audio-Visual Speech Recognition with Minimal Multimodal Speech Tokens
par: Yeo, Jeong Hun, et autres
Publié: (2025)

A Transcription Prompt-based Efficient Audio Large Language Model for Robust Speech Recognition
par: Li, Yangze, et autres
Publié: (2024)

Efficient Training for Multilingual Visual Speech Recognition: Pre-training with Discretized Visual Speech Representation
par: Kim, Minsu, et autres
Publié: (2024)

ThinkSound: Chain-of-Thought Reasoning in Multimodal Large Language Models for Audio Generation and Editing
par: Liu, Huadai, et autres
Publié: (2025)

EmoQ: Speech Emotion Recognition via Speech-Aware Q-Former and Large Language Model
par: Yang, Yiqing, et autres
Publié: (2025)

Streaming Speech Recognition with Decoder-Only Large Language Models and Latency Optimization
par: Wan, Genshun, et autres
Publié: (2026)

CNVSRC 2024: The Second Chinese Continuous Visual Speech Recognition Challenge
par: Liu, Zehua, et autres
Publié: (2025)

SSAVSV: Towards Unified Model for Self-Supervised Audio-Visual Speaker Verification
par: Rajasekhar, Gnana Praveen, et autres
Publié: (2025)

mWhisper-Flamingo for Multilingual Audio-Visual Noise-Robust Speech Recognition
par: Rouditchenko, Andrew, et autres
Publié: (2025)