Imagen de Portada

Why Do Speech Language Models Fail to Generate Semantically Coherent Outputs? A Modality Evolving Perspective

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wang, Hankun, Wang, Haoran, Guo, Yiwei, Li, Zhihan, Du, Chenpeng, Yu, Kai
Formato:	Preprint
Publicado:	2024
Materias:	Audio and Speech Processing Computation and Language Sound
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

LSCodec: Low-Bitrate and Speaker-Decoupled Discrete Speech Codec
por: Guo, Yiwei, et al.
Publicado: (2024)

Attention-Constrained Inference for Robust Decoder-Only Text-to-Speech
por: Wang, Hankun, et al.
Publicado: (2024)

VALL-T: Decoder-Only Generative Transducer for Robust and Decoding-Controllable Text-to-Speech
por: Du, Chenpeng, et al.
Publicado: (2024)

Towards General Discrete Speech Codec for Complex Acoustic Environments: A Study of Reconstruction and Downstream Task Consistency
por: Wang, Haoran, et al.
Publicado: (2025)

Acoustic BPE for Speech Generation with Discrete Tokens
por: Shen, Feiyu, et al.
Publicado: (2023)

VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Feature
por: Du, Chenpeng, et al.
Publicado: (2022)

vec2wav 2.0: Advancing Voice Conversion via Discrete Token Vocoders
por: Guo, Yiwei, et al.
Publicado: (2024)

Robust and Efficient Autoregressive Speech Synthesis with Dynamic Chunk-wise Prediction Policy
por: Li, Bohan, et al.
Publicado: (2025)

Phone-Level Prosody Modelling with GMM-Based MDN for Diverse and Controllable Speech Synthesis
por: Du, Chenpeng, et al.
Publicado: (2021)

CodecSlime: Temporal Redundancy Compression of Neural Speech Codec via Dynamic Frame Rate
por: Wang, Hankun, et al.
Publicado: (2025)

Multi-Speaker Multi-Lingual VQTTS System for LIMMITS 2023 Challenge
por: Du, Chenpeng, et al.
Publicado: (2023)

Recent Advances in Discrete Speech Tokens: A Review
por: Guo, Yiwei, et al.
Publicado: (2025)

AHAMask: Reliable Task Specification for Large Audio Language Models without Instructions
por: Guo, Yiwei, et al.
Publicado: (2025)

HoliTok:A Coutinuous Holistic Tokenization with Robust Dual Capabilities of Speech Generation and Understanding
por: Li, Bohan, et al.
Publicado: (2026)

UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocoding
por: Du, Chenpeng, et al.
Publicado: (2023)

Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decoding
por: Li, Bohan, et al.
Publicado: (2024)

VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching
por: Guo, Yiwei, et al.
Publicado: (2023)

ISA-Bench: Benchmarking Instruction Sensitivity for Large Audio Language Models
por: Li, Bohan, et al.
Publicado: (2025)

Unlocking Temporal Flexibility: Neural Speech Codec with Variable Frame Rate
por: Zhang, Hanglei, et al.
Publicado: (2025)

SoCodec: A Semantic-Ordered Multi-Stream Speech Codec for Efficient Language Model Based Text-to-Speech Synthesis
por: Guo, Haohan, et al.
Publicado: (2024)

Music2Fail: Transfer Music to Failed Recorder Style
por: Leong, Chon In, et al.
Publicado: (2024)

Investigating Neural Audio Codecs for Speech Language Model-Based Speech Generation
por: Li, Jiaqi, et al.
Publicado: (2024)

In This Environment, As That Speaker: A Text-Driven Framework for Multi-Attribute Speech Conversion
por: Jin, Jiawei, et al.
Publicado: (2025)

DualCodec: A Low-Frame-Rate, Semantically-Enhanced Neural Audio Codec for Speech Generation
por: Li, Jiaqi, et al.
Publicado: (2025)

Mixture of LoRA Experts with Multi-Modal and Multi-Granularity LLM Generative Error Correction for Accented Speech Recognition
por: Mu, Bingshen, et al.
Publicado: (2025)

DualSpeechLM: Towards Unified Speech Understanding and Generation via Dual Speech Token Modeling with Large Language Models
por: Wang, Yuanyuan, et al.
Publicado: (2025)

Accelerating Diffusion-based Text-to-Speech Model Training with Dual Modality Alignment
por: Choi, Jeongsoo, et al.
Publicado: (2025)

StoryTTS: A Highly Expressive Text-to-Speech Dataset with Rich Textual Expressiveness Annotations
por: Liu, Sen, et al.
Publicado: (2024)

Exploiting Audio-Visual Features with Pretrained AV-HuBERT for Multi-Modal Dysarthric Speech Reconstruction
por: Chen, Xueyuan, et al.
Publicado: (2024)

Why Your Tokenizer Fails in Information Fusion: A Timing-Aware Pre-Quantization Fusion for Video-Enhanced Audio Tokenization
por: Zhang, Xiangyu, et al.
Publicado: (2026)

Evaluating Text-to-Speech Synthesis from a Large Discrete Token-based Speech Language Model
por: Wang, Siyang, et al.
Publicado: (2024)

SECodec: Structural Entropy-based Compressive Speech Representation Codec for Speech Language Models
por: Wang, Linqin, et al.
Publicado: (2024)

FireRedTTS: A Foundation Text-To-Speech Framework for Industry-Level Generative Speech Applications
por: Guo, Hao-Han, et al.
Publicado: (2024)

Combined Generative and Predictive Modeling for Speech Super-resolution
por: Wang, Heming, et al.
Publicado: (2024)

FleSpeech: Flexibly Controllable Speech Generation with Various Prompts
por: Li, Hanzhao, et al.
Publicado: (2025)

CAMEL: Cross-Attention Enhanced Mixture-of-Experts and Language Bias for Code-Switching Speech Recognition
por: Wang, He, et al.
Publicado: (2024)

Language-Codec: Bridging Discrete Codec Representations and Speech Language Models
por: Ji, Shengpeng, et al.
Publicado: (2024)

DialoSpeech: Dual-Speaker Dialogue Generation with LLM and Flow Matching
por: Xie, Hanke, et al.
Publicado: (2025)

Amphion: An Open-Source Audio, Music and Speech Generation Toolkit
por: Zhang, Xueyao, et al.
Publicado: (2023)

DiveSound: LLM-Assisted Automatic Taxonomy Construction for Diverse Audio Generation
por: Li, Baihan, et al.
Publicado: (2024)