Imagen de Portada

Self-Powered LLM Modality Expansion for Large Speech-Text Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yu, Tengfei, Liu, Xuebo, Hou, Zhiyi, Ding, Liang, Tao, Dacheng, Zhang, Min
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Sound Audio and Speech Processing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Adaptive Inner Speech-Text Alignment for LLM-based Speech Translation
por: Liu, Henglyu, et al.
Publicado: (2025)

Closing the Modality Reasoning Gap for Speech Large Language Models
por: Wang, Chaoren, et al.
Publicado: (2026)

Speech Recognition Rescoring with Large Speech-Text Foundation Models
por: Shivakumar, Prashanth Gurunath, et al.
Publicado: (2024)

Minimizing Modality Gap from the Input Side: Your Speech LLM Can Be a Prosody-Aware Text LLM
por: Cui, Wenqian, et al.
Publicado: (2026)

Improving Audio Codec-based Zero-Shot Text-to-Speech Synthesis with Multi-Modal Context and Large Language Model
por: Xue, Jinlong, et al.
Publicado: (2024)

AzeroS: Extending LLM to Speech with Self-Generated Instruction-Free Tuning
por: Shao, Yiwen, et al.
Publicado: (2025)

Enhancing Generalization of Speech Large Language Models with Multi-Task Behavior Imitation and Speech-Text Interleaving
por: Xie, Jingran, et al.
Publicado: (2025)

Generating Data with Text-to-Speech and Large-Language Models for Conversational Speech Recognition
por: Cornell, Samuele, et al.
Publicado: (2024)

Continuous Speech Tokenizer in Text To Speech
por: Li, Yixing, et al.
Publicado: (2024)

S2SBench: A Benchmark for Quantifying Intelligence Degradation in Speech-to-Speech Large Language Models
por: Fang, Yuanbo, et al.
Publicado: (2025)

TI-ASU: Toward Robust Automatic Speech Understanding through Text-to-speech Imputation Against Missing Speech Modality
por: Feng, Tiantian, et al.
Publicado: (2024)

Textless Acoustic Model with Self-Supervised Distillation for Noise-Robust Expressive Speech-to-Speech Translation
por: Hwang, Min-Jae, et al.
Publicado: (2024)

Exploring the Integration of Large Language Models into Automatic Speech Recognition Systems: An Empirical Study
por: Min, Zeping, et al.
Publicado: (2023)

TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
por: Tseng, Liang-Hsuan, et al.
Publicado: (2025)

LESS: Large Language Model Enhanced Semi-Supervised Learning for Speech Foundational Models Using in-the-wild Data
por: Ding, Wen, et al.
Publicado: (2025)

SSR: Alignment-Aware Modality Connector for Speech Language Models
por: Tan, Weiting, et al.
Publicado: (2024)

SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
por: Zhang, Xin, et al.
Publicado: (2023)

Analytic Study of Text-Free Speech Synthesis for Raw Audio using a Self-Supervised Learning Model
por: Park, Joonyong, et al.
Publicado: (2024)

Enhancing Code-switched Text-to-Speech Synthesis Capability in Large Language Models with only Monolingual Corpora
por: Xu, Jing, et al.
Publicado: (2024)

Leveraging LLM and Self-Supervised Training Models for Speech Recognition in Chinese Dialects: A Comparative Analysis
por: Xu, Tianyi, et al.
Publicado: (2025)

Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis
por: Zhou, Kun, et al.
Publicado: (2024)

Scaling Analysis of Interleaved Speech-Text Language Models
por: Maimon, Gallil, et al.
Publicado: (2025)

Harnessing the Zero-Shot Power of Instruction-Tuned Large Language Model in End-to-End Speech Recognition
por: Higuchi, Yosuke, et al.
Publicado: (2023)

Fast Word Error Rate Estimation Using Self-Supervised Representations for Speech and Text
por: Park, Chanho, et al.
Publicado: (2023)

Scheduled Interleaved Speech-Text Training for Speech-to-Speech Translation with LLMs
por: Futami, Hayato, et al.
Publicado: (2025)

StreamUni: Achieving Streaming Speech Translation with a Unified Large Speech-Language Model
por: Guo, Shoutao, et al.
Publicado: (2025)

Interface Design for Self-Supervised Speech Models
por: Shih, Yi-Jen, et al.
Publicado: (2024)

SimpleSpeech 2: Towards Simple and Efficient Text-to-Speech with Flow-based Scalar Latent Transformer Diffusion Models
por: Yang, Dongchao, et al.
Publicado: (2024)

Exploring Effective Distillation of Self-Supervised Speech Models for Automatic Speech Recognition
por: Wang, Yujin, et al.
Publicado: (2022)

Joint Speech and Text Training for LLM-Based End-to-End Spoken Dialogue State Tracking
por: Vendrame, Katia, et al.
Publicado: (2025)

Why Do Speech Language Models Fail to Generate Semantically Coherent Outputs? A Modality Evolving Perspective
por: Wang, Hankun, et al.
Publicado: (2024)

StoryTTS: A Highly Expressive Text-to-Speech Dataset with Rich Textual Expressiveness Annotations
por: Liu, Sen, et al.
Publicado: (2024)

Emphasis Rendering for Conversational Text-to-Speech with Multi-modal Multi-scale Context Modeling
por: Liu, Rui, et al.
Publicado: (2024)

On the Effects of Heterogeneous Data Sources on Speech-to-Text Foundation Models
por: Tian, Jinchuan, et al.
Publicado: (2024)

SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large Language Models
por: Peri, Raghuveer, et al.
Publicado: (2024)

SEAL: Speech Embedding Alignment Learning for Speech Large Language Model with Retrieval-Augmented Generation
por: Sun, Chunyu, et al.
Publicado: (2025)

Accelerating Diffusion-based Text-to-Speech Model Training with Dual Modality Alignment
por: Choi, Jeongsoo, et al.
Publicado: (2025)

Paralinguistics-Aware Speech-Empowered Large Language Models for Natural Conversation
por: Kim, Heeseung, et al.
Publicado: (2024)

Position-invariant Fine-tuning of Speech Enhancement Models with Self-supervised Speech Representations
por: Meghanani, Amit, et al.
Publicado: (2026)

DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment
por: Lu, Ke-Han, et al.
Publicado: (2025)