Cover Image

SSR: Alignment-Aware Modality Connector for Speech Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tan, Weiting, Inaguma, Hirofumi, Dong, Ning, Tomasello, Paden, Ma, Xutai
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Sound Audio and Speech Processing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Seeing is Believing: Emotion-Aware Audio-Visual Language Modeling for Expressive Speech Generation
by: Tan, Weiting, et al.
Published: (2025)

MMM: Multi-Layer Multi-Residual Multi-Stream Discrete Speech Representation from Self-supervised Learning Model
by: Shi, Jiatong, et al.
Published: (2024)

Multi-resolution HuBERT: Multi-resolution Speech Self-Supervised Learning with Masked Unit Prediction
by: Shi, Jiatong, et al.
Published: (2023)

Investigating Decoder-only Large Language Models for Speech-to-text Translation
by: Huang, Chao-Wei, et al.
Published: (2024)

Closing the Modality Reasoning Gap for Speech Large Language Models
by: Wang, Chaoren, et al.
Published: (2026)

LAST: Language Model Aware Speech Tokenization
by: Turetzky, Arnon, et al.
Published: (2024)

SEAL: Speech Embedding Alignment Learning for Speech Large Language Model with Retrieval-Augmented Generation
by: Sun, Chunyu, et al.
Published: (2025)

Minimizing Modality Gap from the Input Side: Your Speech LLM Can Be a Prosody-Aware Text LLM
by: Cui, Wenqian, et al.
Published: (2026)

SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
by: Zhang, Xin, et al.
Published: (2023)

Paralinguistics-Aware Speech-Empowered Large Language Models for Natural Conversation
by: Kim, Heeseung, et al.
Published: (2024)

Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis
by: Zhou, Kun, et al.
Published: (2024)

Why Do Speech Language Models Fail to Generate Semantically Coherent Outputs? A Modality Evolving Perspective
by: Wang, Hankun, et al.
Published: (2024)

Self-Powered LLM Modality Expansion for Large Speech-Text Models
by: Yu, Tengfei, et al.
Published: (2024)

Improving Audio Codec-based Zero-Shot Text-to-Speech Synthesis with Multi-Modal Context and Large Language Model
by: Xue, Jinlong, et al.
Published: (2024)

Generative Pre-trained Speech Language Model with Efficient Hierarchical Transformer
by: Zhu, Yongxin, et al.
Published: (2024)

Pinyin Regularization in Error Correction for Chinese Speech Recognition with Large Language Models
by: Tang, Zhiyuan, et al.
Published: (2024)

Adaptive Inner Speech-Text Alignment for LLM-based Speech Translation
by: Liu, Henglyu, et al.
Published: (2025)

BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writing
by: Wang, Chen, et al.
Published: (2023)

ALAS: Measuring Latent Speech-Text Alignment For Spoken Language Understanding In Multimodal LLMs
by: Mousavi, Pooneh, et al.
Published: (2025)

Generalized Multilingual Text-to-Speech Generation with Language-Aware Style Adaptation
by: Lou, Haowei, et al.
Published: (2025)

Enhancing Code-switched Text-to-Speech Synthesis Capability in Large Language Models with only Monolingual Corpora
by: Xu, Jing, et al.
Published: (2024)

ESPnet-SpeechLM: An Open Speech Language Model Toolkit
by: Tian, Jinchuan, et al.
Published: (2025)

SALM-Duplex: Efficient and Direct Duplex Modeling for Speech-to-Speech Language Model
by: Hu, Ke, et al.
Published: (2025)

DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment
by: Lu, Ke-Han, et al.
Published: (2025)

Multi-Teacher Language-Aware Knowledge Distillation for Multilingual Speech Emotion Recognition
by: Bijoy, Mehedi Hasan, et al.
Published: (2025)

An Empirical Study of Speech Language Models for Prompt-Conditioned Speech Synthesis
by: Peng, Yifan, et al.
Published: (2024)

Efficient Speech Language Modeling via Energy Distance in Continuous Latent Space
by: Ma, Zhengrui, et al.
Published: (2025)

Assessment of L2 Oral Proficiency using Speech Large Language Models
by: Ma, Rao, et al.
Published: (2025)

Generating Data with Text-to-Speech and Large-Language Models for Conversational Speech Recognition
by: Cornell, Samuele, et al.
Published: (2024)

Dynamic Context-Aware Streaming Pretrained Language Model For Inverse Text Normalization
by: Ho, Luong, et al.
Published: (2025)

Exploring the Effect of Segmentation and Vocabulary Size on Speech Tokenization for Speech Language Models
by: Kando, Shunsuke, et al.
Published: (2025)

Coarse-to-fine Alignment Makes Better Speech-image Retrieval
by: Zhou, Lifeng, et al.
Published: (2024)

Customizing Speech Recognition Model with Large Language Model Feedback
by: Ling, Shaoshi, et al.
Published: (2025)

MSLM-S2ST: A Multitask Speech Language Model for Textless Speech-to-Speech Translation with Speaker Style Preservation
by: Peng, Yifan, et al.
Published: (2024)

Speech Language Models for Under-Represented Languages: Insights from Wolof
by: Sy, Yaya, et al.
Published: (2025)

Sagalee: an Open Source Automatic Speech Recognition Dataset for Oromo Language
by: Abu, Turi, et al.
Published: (2025)

Large Language Models for Dysfluency Detection in Stuttered Speech
by: Wagner, Dominik, et al.
Published: (2024)

Recent Advances in Speech Language Models: A Survey
by: Cui, Wenqian, et al.
Published: (2024)

On the Evaluation of Speech Foundation Models for Spoken Language Understanding
by: Arora, Siddhant, et al.
Published: (2024)

Long-Form Speech Generation with Spoken Language Models
by: Park, Se Jin, et al.
Published: (2024)