Copertina

AlignFormer: Modality Matching Can Achieve Better Zero-shot Instruction-Following Speech-LLM

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Fan, Ruchao, Ren, Bo, Hu, Yuxuan, Zhao, Rui, Liu, Shujie, Li, Jinyu
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Audio and Speech Processing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Train Short, Infer Long: Speech-LLM Enables Zero-Shot Streamable Joint ASR and Diarization on Long Audio
di: Shi, Mohan, et al.
Pubblicazione: (2025)

RLBR: Reinforcement Learning with Biasing Rewards for Contextual Speech Large Language Models
di: Ren, Bo, et al.
Pubblicazione: (2026)

Towards Efficient Speech-Text Jointly Decoding within One Speech Language Model
di: Wu, Haibin, et al.
Pubblicazione: (2025)

SLM-S2ST: A multimodal language model for direct speech-to-speech translation
di: Hu, Yuxuan, et al.
Pubblicazione: (2025)

CTC-GMM: CTC guided modality matching for fast and accurate streaming speech translation
di: Zhao, Rui, et al.
Pubblicazione: (2024)

SOA: Reducing Domain Mismatch in SSL Pipeline by Speech Only Adaptation for Low Resource ASR
di: Shankar, Natarajan Balaji, et al.
Pubblicazione: (2024)

Boosting Large Language Model for Speech Synthesis: An Empirical Study
di: Hao, Hongkun, et al.
Pubblicazione: (2023)

Interleaved Speech-Text Language Models for Simple Streaming Text-to-Speech Synthesis
di: Yang, Yifan, et al.
Pubblicazione: (2024)

StreamMel: Real-Time Zero-shot Text-to-Speech via Interleaved Continuous Autoregressive Modeling
di: Wang, Hui, et al.
Pubblicazione: (2025)

On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognition
di: HU, Shujie, et al.
Pubblicazione: (2025)

Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models
di: Fan, Ruchao, et al.
Pubblicazione: (2024)

Closing the Modality Reasoning Gap for Speech Large Language Models
di: Wang, Chaoren, et al.
Pubblicazione: (2026)

Advanced Long-Content Speech Recognition With Factorized Neural Transducer
di: Gong, Xun, et al.
Pubblicazione: (2024)

WHISMA: A Speech-LLM to Perform Zero-shot Spoken Language Understanding
di: Li, Mohan, et al.
Pubblicazione: (2024)

UniEnc-CASSNAT: An Encoder-only Non-autoregressive ASR for Speech SSL Models
di: Fan, Ruchao, et al.
Pubblicazione: (2024)

Pseudo-Autoregressive Neural Codec Language Models for Efficient Zero-Shot Text-to-Speech Synthesis
di: Yang, Yifan, et al.
Pubblicazione: (2025)

Semantic-VAE: Semantic-Alignment Latent Representation for Better Speech Synthesis
di: Niu, Zhikang, et al.
Pubblicazione: (2025)

Position: Towards Responsible Evaluation for Text-to-Speech
di: Yang, Yifan, et al.
Pubblicazione: (2025)

SpeechLLM-as-Judges: Towards General and Interpretable Speech Quality Evaluation
di: Wang, Hui, et al.
Pubblicazione: (2025)

COSMIC: Data Efficient Instruction-tuning For Speech In-Context Learning
di: Pan, Jing, et al.
Pubblicazione: (2023)

SSR-Speech: Towards Stable, Safe and Robust Zero-shot Text-based Speech Editing and Synthesis
di: Wang, Helin, et al.
Pubblicazione: (2024)

Latent Filling: Latent Space Data Augmentation for Zero-shot Speech Synthesis
di: Bae, Jae-Sung, et al.
Pubblicazione: (2023)

Wav2Prompt: End-to-End Speech Prompt Generation and Tuning For LLM in Zero and Few-shot Learning
di: Deng, Keqi, et al.
Pubblicazione: (2024)

Fine-grained Preference Optimization Improves Zero-shot Text-to-Speech
di: Yao, Jixun, et al.
Pubblicazione: (2025)

OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching
di: Huynh-Nguyen, Hieu-Nghia, et al.
Pubblicazione: (2025)

S2S-Arena: Evaluating Paralinguistic Instruction Following in Speech-to-Speech Models
di: Jiang, Feng, et al.
Pubblicazione: (2025)

MINT-Bench: A Comprehensive Multilingual Benchmark for Instruction-Following Text-to-Speech
di: Chen, Huakang, et al.
Pubblicazione: (2026)

Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedback
di: Chen, Chen, et al.
Pubblicazione: (2024)

JASTIN: Aligning LLMs for Zero-Shot Audio and Speech Evaluation via Natural Language Instructions
di: Zhang, Leying, et al.
Pubblicazione: (2026)

WavLLM: Towards Robust and Adaptive Speech Large Language Model
di: Hu, Shujie, et al.
Pubblicazione: (2024)

Continuous Speech Tokens Makes LLMs Robust Multi-Modality Learners
di: Yuan, Ze, et al.
Pubblicazione: (2024)

VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers
di: Chen, Sanyuan, et al.
Pubblicazione: (2024)

DialoSpeech: Dual-Speaker Dialogue Generation with LLM and Flow Matching
di: Xie, Hanke, et al.
Pubblicazione: (2025)

ZMM-TTS: Zero-shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-supervised Discrete Speech Representations
di: Gong, Cheng, et al.
Pubblicazione: (2023)

LiveSpeech: Low-Latency Zero-shot Text-to-Speech via Autoregressive Modeling of Audio Discrete Codes
di: Dang, Trung, et al.
Pubblicazione: (2024)

Large Language Model Can Transcribe Speech in Multi-Talker Scenarios with Versatile Instructions
di: Meng, Lingwei, et al.
Pubblicazione: (2024)

Can We Estimate Purchase Intention Based on Zero-shot Speech Emotion Recognition?
di: Nagase, Ryotaro, et al.
Pubblicazione: (2024)

FELLE: Autoregressive Speech Synthesis with Token-Wise Coarse-to-Fine Flow Matching
di: Wang, Hui, et al.
Pubblicazione: (2025)

UniTalker: Conversational Speech-Visual Synthesis
di: Hu, Yifan, et al.
Pubblicazione: (2025)

Autoregressive Speech Synthesis without Vector Quantization
di: Meng, Lingwei, et al.
Pubblicazione: (2024)