Copertina

DubWise: Video-Guided Speech Duration Control in Multimodal LLM-based Text-to-Speech for Dubbing

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Sahipjohn, Neha, Gudmalwar, Ashishkumar, Shah, Nirmesh, Wasnik, Pankaj, Shah, Rajiv Ratn
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Audio and Speech Processing Sound
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

VECL-TTS: Voice identity and Emotional style controllable Cross-Lingual Text-to-Speech
di: Gudmalwar, Ashishkumar, et al.
Pubblicazione: (2024)

Can Hierarchical Cross-Modal Fusion Predict Human Perception of AI Dubbed Content?
di: Dasare, Ashwini, et al.
Pubblicazione: (2026)

EmoReg: Directional Latent Vector Modeling for Emotional Intensity Regularization in Diffusion-based Voice Conversion
di: Gudmalwar, Ashishkumar, et al.
Pubblicazione: (2024)

REWIND: Speech Time Reversal for Enhancing Speaker Representations in Diffusion-based Voice Conversion
di: Biyani, Ishan D., et al.
Pubblicazione: (2025)

Gesture2Speech: How Far Can Hand Movements Shape Expressive Speech?
di: Kumar, Lokesh, et al.
Pubblicazione: (2026)

Isometric Neural Machine Translation using Phoneme Count Ratio Reward-based Reinforcement Learning
di: Mhaskar, Shivam Ratnakant, et al.
Pubblicazione: (2024)

Dub-S2ST: Textless Speech-to-Speech Translation for Seamless Dubbing
di: Choi, Jeongsoo, et al.
Pubblicazione: (2025)

Analysing the Masked predictive coding training criterion for pre-training a Speech Representation Model
di: Yadav, Hemant, et al.
Pubblicazione: (2023)

Length Aware Speech Translation for Video Dubbing
di: Chadha, Harveen Singh, et al.
Pubblicazione: (2025)

DualDub: Video-to-Soundtrack Generation via Joint Speech and Background Audio Synthesis
di: Tian, Wenjie, et al.
Pubblicazione: (2025)

Towards Expressive Video Dubbing with Multiscale Multimodal Context Interaction
di: Zhao, Yuan, et al.
Pubblicazione: (2024)

MCDubber: Multimodal Context-Aware Expressive Video Dubbing
di: Zhao, Yuan, et al.
Pubblicazione: (2024)

EmoDubber: Towards High Quality and Emotion Controllable Movie Dubbing
di: Cong, Gaoxiang, et al.
Pubblicazione: (2024)

Joint Multi-scale Cross-lingual Speaking Style Transfer with Bidirectional Attention Mechanism for Automatic Dubbing
di: Li, Jingbei, et al.
Pubblicazione: (2023)

Windowed SummaryMixing: An Efficient Fine-Tuning of Self-Supervised Learning Models for Low-resource Speech Recognition
di: Menon, Aditya Srinivas, et al.
Pubblicazione: (2026)

Adaptive Duration Model for Text Speech Alignment
di: Cao, Junjie
Pubblicazione: (2025)

Frame-Wise Breath Detection with Self-Training: An Exploration of Enhancing Breath Naturalness in Text-to-Speech
di: Yang, Dong, et al.
Pubblicazione: (2024)

Adaptive Inner Speech-Text Alignment for LLM-based Speech Translation
di: Liu, Henglyu, et al.
Pubblicazione: (2025)

Beyond Speaker Identity: Text Guided Target Speech Extraction
di: Huo, Mingyue, et al.
Pubblicazione: (2025)

Advanced Zero-Shot Text-to-Speech for Background Removal and Preservation with Controllable Masked Speech Prediction
di: Zhang, Leying, et al.
Pubblicazione: (2025)

Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition
di: Bai, Ye, et al.
Pubblicazione: (2024)

Towards Improving NAM-to-Speech Synthesis Intelligibility using Self-Supervised Speech Models
di: Shah, Neil, et al.
Pubblicazione: (2024)

Hierarchical Emotion Prediction and Control in Text-to-Speech Synthesis
di: Inoue, Sho, et al.
Pubblicazione: (2024)

Inter-Speaker Relative Cues for Text-Guided Target Speech Extraction
di: Dai, Wang, et al.
Pubblicazione: (2025)

FlowDubber: Movie Dubbing with LLM-based Semantic-aware Learning and Flow Matching based Voice Enhancing
di: Cong, Gaoxiang, et al.
Pubblicazione: (2025)

TraceableSpeech: Towards Proactively Traceable Text-to-Speech with Watermarking
di: Zhou, Junzuo, et al.
Pubblicazione: (2024)

Ideal-LLM: Integrating Dual Encoders and Language-Adapted LLM for Multilingual Speech-to-Text
di: Xue, Hongfei, et al.
Pubblicazione: (2024)

SSR-Speech: Towards Stable, Safe and Robust Zero-shot Text-based Speech Editing and Synthesis
di: Wang, Helin, et al.
Pubblicazione: (2024)

Evaluating Text-to-Speech Synthesis from a Large Discrete Token-based Speech Language Model
di: Wang, Siyang, et al.
Pubblicazione: (2024)

Personalized Fine-Tuning with Controllable Synthetic Speech from LLM-Generated Transcripts for Dysarthric Speech Recognition
di: Wagner, Dominik, et al.
Pubblicazione: (2025)

A Multi-Stage Framework for Multimodal Controllable Speech Synthesis
di: Niu, Rui, et al.
Pubblicazione: (2025)

Zero Shot Text to Speech Augmentation for Automatic Speech Recognition on Low-Resource Accented Speech Corpora
di: Nespoli, Francesco, et al.
Pubblicazione: (2024)

Advancing Electrolaryngeal Speech Enhancement Through Speech-Text Representation Learning
di: Ma, Ding, et al.
Pubblicazione: (2026)

DurIAN-E 2: Duration Informed Attention Network with Adaptive Variational Autoencoder and Adversarial Learning for Expressive Text-to-Speech Synthesis
di: Gu, Yu, et al.
Pubblicazione: (2024)

MRI2Speech: Speech Synthesis from Articulatory Movements Recorded by Real-time MRI
di: Shah, Neil, et al.
Pubblicazione: (2024)

TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Models
di: Ji, Shengpeng, et al.
Pubblicazione: (2023)

LipVoicer: Generating Speech from Silent Videos Guided by Lip Reading
di: Yemini, Yochai, et al.
Pubblicazione: (2023)

S2ST-Omni: Hierarchical Language-Aware SpeechLLM Adaptation for Multilingual Speech-to-Speech Translation
di: Pan, Yu, et al.
Pubblicazione: (2025)

SpeechLLM-as-Judges: Towards General and Interpretable Speech Quality Evaluation
di: Wang, Hui, et al.
Pubblicazione: (2025)

PS-TTS: Phonetic Synchronization in Text-to-Speech for Achieving Natural Automated Dubbing
di: Hong, Changi, et al.
Pubblicazione: (2026)