Copertina

Discrete Multimodal Transformers with a Pretrained Large Language Model for Mixed-Supervision Speech Processing

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Trinh, Viet Anh, Southwell, Rosy, Guan, Yiwen, He, Xinlu, Wang, Zhiyong, Whitehill, Jacob
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Machine Learning Audio and Speech Processing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

MLLM-based Speech Recognition: When and How is Multimodality Beneficial?
di: Guan, Yiwen, et al.
Pubblicazione: (2025)

Multi-modal Speech Transformer Decoders: When Do Multiple Modalities Improve Accuracy?
di: Guan, Yiwen, et al.
Pubblicazione: (2024)

Survey of End-to-End Multi-Speaker Automatic Speech Recognition for Monaural Audio
di: He, Xinlu, et al.
Pubblicazione: (2025)

Reducing Geographic Disparities in Automatic Speech Recognition via Elastic Weight Consolidation
di: Trinh, Viet Anh, et al.
Pubblicazione: (2022)

DualSpeechLM: Towards Unified Speech Understanding and Generation via Dual Speech Token Modeling with Large Language Models
di: Wang, Yuanyuan, et al.
Pubblicazione: (2025)

Large Language Model Guided Decoding for Self-Supervised Speech Recognition
di: Cohen, Eyal, et al.
Pubblicazione: (2025)

Two-pass Endpoint Detection for Speech Recognition
di: Raju, Anirudh, et al.
Pubblicazione: (2024)

Adapting Speech Foundation Models for Unified Multimodal Speech Recognition with Large Language Models
di: Zhang, Jing-Xuan, et al.
Pubblicazione: (2025)

Rethinking Mamba in Speech Processing by Self-Supervised Models
di: Zhang, Xiangyu, et al.
Pubblicazione: (2024)

Benchmarking Large Pretrained Multilingual Models on Québec French Speech Recognition
di: Serrand, Coralie, et al.
Pubblicazione: (2025)

DiscreteSLU: A Large Language Model with Self-Supervised Discrete Speech Units for Spoken Language Understanding
di: Shon, Suwon, et al.
Pubblicazione: (2024)

Leveraging Self-Supervised Audio-Visual Pretrained Models to Improve Vocoded Speech Intelligibility in Cochlear Implant Simulation
di: Lai, Richard Lee, et al.
Pubblicazione: (2023)

Unsupervised Accent Adaptation Through Masked Language Model Correction Of Discrete Self-Supervised Speech Units
di: Poncelet, Jakob, et al.
Pubblicazione: (2023)

Evaluating Text-to-Speech Synthesis from a Large Discrete Token-based Speech Language Model
di: Wang, Siyang, et al.
Pubblicazione: (2024)

Compact Speech Translation Models via Discrete Speech Units Pretraining
di: Lam, Tsz Kin, et al.
Pubblicazione: (2024)

A Multi-Stage Framework for Multimodal Controllable Speech Synthesis
di: Niu, Rui, et al.
Pubblicazione: (2025)

The Interspeech 2024 Challenge on Speech Processing Using Discrete Units
di: Chang, Xuankai, et al.
Pubblicazione: (2024)

Exploiting Audio-Visual Features with Pretrained AV-HuBERT for Multi-Modal Dysarthric Speech Reconstruction
di: Chen, Xueyuan, et al.
Pubblicazione: (2024)

Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokens
di: Ku, Pin-Jui, et al.
Pubblicazione: (2025)

XLSR-Transducer: Streaming ASR for Self-Supervised Pretrained Models
di: Kumar, Shashi, et al.
Pubblicazione: (2024)

Efficient Extraction of Noise-Robust Discrete Units from Self-Supervised Speech Models
di: Poncelet, Jakob, et al.
Pubblicazione: (2024)

ToneUnit: A Speech Discretization Approach for Tonal Language Speech Synthesis
di: Tao, Dehua, et al.
Pubblicazione: (2024)

Privacy Disclosure of Similarity Rank in Speech and Language Processing
di: Bäckström, Tom, et al.
Pubblicazione: (2025)

Advances in Microphone Array Processing and Multichannel Speech Enhancement
di: Huang, Gongping, et al.
Pubblicazione: (2025)

SELM: Speech Enhancement Using Discrete Tokens and Language Models
di: Wang, Ziqian, et al.
Pubblicazione: (2023)

UniSRM: A Unified Speech Reward Model for Reasoning-Based Fine-grained Assessment
di: Wang, Yuanyuan, et al.
Pubblicazione: (2026)

Language-Codec: Bridging Discrete Codec Representations and Speech Language Models
di: Ji, Shengpeng, et al.
Pubblicazione: (2024)

RA-CLAP: Relation-Augmented Emotional Speaking Style Contrastive Language-Audio Pretraining For Speech Retrieval
di: Sun, Haoqin, et al.
Pubblicazione: (2025)

Zero-Shot Recognition of Dysarthric Speech Using Commercial Automatic Speech Recognition and Multimodal Large Language Models
di: Alsayegh, Ali, et al.
Pubblicazione: (2025)

TTA: Transcribe, Translate and Alignment for Cross-lingual Speech Representation
di: Liu, Wei, et al.
Pubblicazione: (2025)

Adaptive Endpointing with Deep Contextual Multi-armed Bandits
di: Min, Do June, et al.
Pubblicazione: (2023)

UTDUSS: UTokyo-SaruLab System for Interspeech2024 Speech Processing Using Discrete Speech Unit Challenge
di: Nakata, Wataru, et al.
Pubblicazione: (2024)

Sub-band Domain Multi-Hypothesis Acoustic Echo Canceler Based Acoustic Scene Analysis
di: Southwell, Benjamin J, et al.
Pubblicazione: (2025)

EmoSSLSphere: Multilingual Emotional Speech Synthesis with Spherical Vectors and Discrete Speech Tokens
di: Park, Joonyong, et al.
Pubblicazione: (2025)

Spoken-Term Discovery using Discrete Speech Units
di: van Niekerk, Benjamin, et al.
Pubblicazione: (2024)

A Survey on Speech Large Language Models for Understanding
di: Peng, Jing, et al.
Pubblicazione: (2024)

MixedG2P-T5: G2P-free Speech Synthesis for Mixed-script texts using Speech Self-Supervised Learning and Language Model
di: Park, Joonyong, et al.
Pubblicazione: (2025)

Auden-Voice: General-Purpose Voice Encoder for Speech and Language Understanding
di: Huo, Mingyue, et al.
Pubblicazione: (2025)

Device Feature based on Graph Fourier Transformation with Logarithmic Processing For Detection of Replay Speech Attacks
di: He, Mingrui, et al.
Pubblicazione: (2024)

Textually Pretrained Speech Language Models
di: Hassid, Michael, et al.
Pubblicazione: (2023)