Image de couverture de livre

SynesLM: A Unified Approach for Audio-visual Speech Recognition and Translation via Language Model and Synthetic Data

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Lu, Yichen, Song, Jiaqi, Chang, Xuankai, Bian, Hengwei, Maiti, Soumi, Watanabe, Shinji
Format:	Preprint
Publié:	2024
Sujets:	Audio and Speech Processing Computation and Language Computer Vision and Pattern Recognition
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Robust Audiovisual Speech Recognition Models with Mixture-of-Experts
par: Wu, Yihan, et autres
Publié: (2024)

TMT: Tri-Modal Translation between Speech, Image, and Text by Processing Different Modalities as Different Languages
par: Kim, Minsu, et autres
Publié: (2024)

Towards Robust Speech Representation Learning for Thousands of Languages
par: Chen, William, et autres
Publié: (2024)

SpeechComposer: Unifying Multiple Speech Tasks with Prompt Composition
par: Wu, Yihan, et autres
Publié: (2024)

SpeechBERTScore: Reference-Aware Automatic Evaluation of Speech Generation Leveraging NLP Evaluation Metrics
par: Saeki, Takaaki, et autres
Publié: (2024)

Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation tasks
par: Maiti, Soumi, et autres
Publié: (2023)

FastAdaSP: Multitask-Adapted Efficient Inference for Large Speech Language Model
par: Lu, Yichen, et autres
Publié: (2024)

OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identification
par: Peng, Yifan, et autres
Publié: (2024)

OpusLM: A Family of Open Unified Speech Language Models
par: Tian, Jinchuan, et autres
Publié: (2025)

Generating Data with Text-to-Speech and Large-Language Models for Conversational Speech Recognition
par: Cornell, Samuele, et autres
Publié: (2024)

Recent Trends in Distant Conversational Speech Recognition: A Review of CHiME-7 and 8 DASR Challenges
par: Cornell, Samuele, et autres
Publié: (2025)

Decoder-only Architecture for Streaming End-to-end Speech Recognition
par: Tsunoo, Emiru, et autres
Publié: (2024)

LV-CTC: Non-autoregressive ASR with CTC and latent variable models
par: Fujita, Yuya, et autres
Publié: (2024)

YODAS: Youtube-Oriented Dataset for Audio and Speech
par: Li, Xinjian, et autres
Publié: (2024)

Scheduled Interleaved Speech-Text Training for Speech-to-Speech Translation with LLMs
par: Futami, Hayato, et autres
Publié: (2025)

Contextualized Automatic Speech Recognition with Dynamic Vocabulary
par: Sudo, Yui, et autres
Publié: (2024)

Contextualized End-to-end Automatic Speech Recognition with Intermediate Biasing Loss
par: Shakeel, Muhammad, et autres
Publié: (2024)

Reasoning Beyond Majority Vote: An Explainable SpeechLM Framework for Speech Emotion Recognition
par: Su, Bo-Hao, et autres
Publié: (2025)

ESPnet-SpeechLM: An Open Speech Language Model Toolkit
par: Tian, Jinchuan, et autres
Publié: (2025)

Leveraging Synthetic Audio Data for End-to-End Low-Resource Speech Translation
par: Moslem, Yasmin
Publié: (2024)

Joint Optimization of Streaming and Non-Streaming Automatic Speech Recognition with Multi-Decoder and Knowledge Distillation
par: Shakeel, Muhammad, et autres
Publié: (2024)

OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer
par: Peng, Yifan, et autres
Publié: (2024)

Hypothesis Clustering and Merging: Novel MultiTalker Speech Recognition with Speaker Tokens
par: Kashiwagi, Yosuke, et autres
Publié: (2024)

OWLS: Scaling Laws for Multilingual Speech Recognition and Translation Models
par: Chen, William, et autres
Publié: (2025)

Contextualized Automatic Speech Recognition with Attention-Based Bias Phrase Boosted Beam Search
par: Sudo, Yui, et autres
Publié: (2024)

Spiralformer: Low Latency Encoder for Streaming Speech Recognition with Circular Layer Skipping and Early Exiting
par: Tsunoo, Emiru, et autres
Publié: (2025)

Rapid Language Adaptation for Multilingual E2E Speech Recognition Using Encoder Prompting
par: Kashiwagi, Yosuke, et autres
Publié: (2024)

ML-SUPERB: Multilingual Speech Universal PERformance Benchmark
par: Shi, Jiatong, et autres
Publié: (2023)

The CHiME-8 DASR Challenge for Generalizable and Array Agnostic Distant Automatic Speech Recognition and Diarization
par: Cornell, Samuele, et autres
Publié: (2024)

Transcribing and Translating, Fast and Slow: Joint Speech Translation and Recognition
par: Moritz, Niko, et autres
Publié: (2024)

DYNAC: Dynamic Vocabulary based Non-Autoregressive Contextualization for Speech Recognition
par: Sudo, Yui, et autres
Publié: (2025)

ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets
par: Shi, Jiatong, et autres
Publié: (2024)

Zero-resource Speech Translation and Recognition with LLMs
par: Mundnich, Karel, et autres
Publié: (2024)

SSVD-O: Parameter-Efficient Fine-Tuning with Structured SVD for Speech Recognition
par: Wang, Pu, et autres
Publié: (2026)

Enhancing Audiovisual Speech Recognition through Bifocal Preference Optimization
par: Wu, Yihan, et autres
Publié: (2024)

Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentation
par: Wu, Shih-Lun, et autres
Publié: (2023)

SQ-Whisper: Speaker-Querying based Whisper Model for Target-Speaker ASR
par: Guo, Pengcheng, et autres
Publié: (2024)

Lightweight Audio Segmentation for Long-form Speech Translation
par: Lee, Jaesong, et autres
Publié: (2024)

Granary: Speech Recognition and Translation Dataset in 25 European Languages
par: Koluguri, Nithin Rao, et autres
Publié: (2025)

POWSM: A Phonetic Open Whisper-Style Speech Foundation Model
par: Li, Chin-Jou, et autres
Publié: (2025)