Buchumschlag

Understanding the Modality Gap: An Empirical Study on the Speech-Text Alignment Mechanism of Large Speech Language Models

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Xiang, Bajian, Zhao, Shuaijiang, Guo, Tingwei, Zou, Wei
Format:	Preprint
Veröffentlicht:	2025
Schlagworte:	Computation and Language Artificial Intelligence
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Ähnliche Einträge

Advancing Speech Language Models by Scaling Supervised Fine-Tuning with Over 60,000 Hours of Synthetic Speech Dialogue Data
von: Zhao, Shuaijiang, et al.
Veröffentlicht: (2024)

Do We Need Distinct Representations for Every Speech Token? Unveiling and Exploiting Redundancy in Large Speech Language Models
von: Xiang, Bajian, et al.
Veröffentlicht: (2026)

Optimal Transport Regularization for Speech Text Alignment in Spoken Language Models
von: Xu, Wenze, et al.
Veröffentlicht: (2025)

Cross-Modal Knowledge Distillation for Speech Large Language Models
von: Wang, Enzhi, et al.
Veröffentlicht: (2025)

Closing the Gap Between Text and Speech Understanding in LLMs
von: Cuervo, Santiago, et al.
Veröffentlicht: (2025)

Preservation of Language Understanding Capabilities in Speech-aware Large Language Models
von: Kubis, Marek, et al.
Veröffentlicht: (2025)

SpeechR: A Benchmark for Speech Reasoning in Large Audio-Language Models
von: Yang, Wanqi, et al.
Veröffentlicht: (2025)

MOSS-Speech: Towards True Speech-to-Speech Models Without Text Guidance
von: Zhao, Xingjian, et al.
Veröffentlicht: (2025)

Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysis
von: Ye, Zongli, et al.
Veröffentlicht: (2025)

Soundwave: Less is More for Speech-Text Alignment in LLMs
von: Zhang, Yuhao, et al.
Veröffentlicht: (2025)

An Empirical Study on Prompt Compression for Large Language Models
von: Zhang, Zheng, et al.
Veröffentlicht: (2025)

SpeechMedAssist: Efficiently and Effectively Adapting Speech Language Models for Medical Consultation
von: Chen, Sirry, et al.
Veröffentlicht: (2026)

SpeechLLM: Unified Speech and Language Model for Enhanced Multi-Task Understanding in Low Resource Settings
von: Yoo, Jaekwon, et al.
Veröffentlicht: (2025)

VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Model
von: Long, Zuwei, et al.
Veröffentlicht: (2025)

Incorporating Contextual Paralinguistic Understanding in Large Speech-Language Models
von: Wang, Qiongqiong, et al.
Veröffentlicht: (2025)

Instituto de Telecomunicações at IWSLT 2025: Aligning Small-Scale Speech and Language Models for Speech-to-Text Learning
von: Attanasio, Giuseppe, et al.
Veröffentlicht: (2025)

Improving Semantic Understanding in Speech Language Models via Brain-tuning
von: Moussa, Omer, et al.
Veröffentlicht: (2024)

Adversarial Attacks on Parts of Speech: An Empirical Study in Text-to-Image Generation
von: Shahariar, G M, et al.
Veröffentlicht: (2024)

Data Generation Using Large Language Models for Text Classification: An Empirical Case Study
von: Li, Yinheng, et al.
Veröffentlicht: (2024)

Cross-Lingual Conversational Speech Summarization with Large Language Models
von: Nelson, Max, et al.
Veröffentlicht: (2024)

Revealing the Numeracy Gap: An Empirical Investigation of Text Embedding Models
von: Deng, Ningyuan, et al.
Veröffentlicht: (2025)

LoASR-Bench: Evaluating Large Speech Language Models on Low-Resource Automatic Speech Recognition Across Language Families
von: Chen, Jianan, et al.
Veröffentlicht: (2026)

Grammar Induction from Visual, Speech and Text
von: Zhao, Yu, et al.
Veröffentlicht: (2024)

Empirical Evaluation of Public HateSpeech Datasets
von: Jaf, Sadar, et al.
Veröffentlicht: (2024)

Text-Printed Image: Bridging the Image-Text Modality Gap for Text-centric Training of Large Vision-Language Models
von: Yamabe, Shojiro, et al.
Veröffentlicht: (2025)

TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignment
von: Li, Wei, et al.
Veröffentlicht: (2024)

Roadmap towards Superhuman Speech Understanding using Large Language Models
von: Bu, Fan, et al.
Veröffentlicht: (2024)

The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models
von: Conti, Lina, et al.
Veröffentlicht: (2025)

PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects
von: Yang, Sicheng, et al.
Veröffentlicht: (2026)

SARI: Structured Audio Reasoning via Curriculum-Guided Reinforcement Learning
von: Wen, Cheng, et al.
Veröffentlicht: (2025)

InfiniSST: Simultaneous Translation of Unbounded Speech with Large Language Model
von: Ouyang, Siqi, et al.
Veröffentlicht: (2025)

Bridging the Data Provenance Gap Across Text, Speech and Video
von: Longpre, Shayne, et al.
Veröffentlicht: (2024)

SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Models
von: Wan, Zhen, et al.
Veröffentlicht: (2025)

Single-to-mix Modality Alignment with Multimodal Large Language Model for Document Image Machine Translation
von: Liang, Yupu, et al.
Veröffentlicht: (2025)

Can Large Language Models Understand DL-Lite Ontologies? An Empirical Study
von: Wang, Keyu, et al.
Veröffentlicht: (2024)

Advancing Speech Understanding in Speech-Aware Language Models with GRPO
von: Elmakies, Avishai, et al.
Veröffentlicht: (2025)

System Report for CCL25-Eval Task 10: Prompt-Driven Large Language Model Merge for Fine-Grained Chinese Hate Speech Detection
von: Wu, Binglin, et al.
Veröffentlicht: (2025)

From Flat Language Labels to Typological Priors: Structured Language Conditioning for Multilingual Speech-to-Speech Translation
von: Pan, Yu, et al.
Veröffentlicht: (2026)

Multi-Modal Retrieval For Large Language Model Based Speech Recognition
von: Kolehmainen, Jari, et al.
Veröffentlicht: (2024)

Conditioning Large Language Models on Legal Systems? Detecting Punishable Hate Speech
von: Ludwig, Florian, et al.
Veröffentlicht: (2025)