Cover Image

Can Hierarchical Cross-Modal Fusion Predict Human Perception of AI Dubbed Content?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dasare, Ashwini, Shah, Nirmesh, Gudmalwar, Ashishkumar, Wasnik, Pankaj
Format:	Preprint
Published:	2026
Subjects:	Audio and Speech Processing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DubWise: Video-Guided Speech Duration Control in Multimodal LLM-based Text-to-Speech for Dubbing
by: Sahipjohn, Neha, et al.
Published: (2024)

Gesture2Speech: How Far Can Hand Movements Shape Expressive Speech?
by: Kumar, Lokesh, et al.
Published: (2026)

VECL-TTS: Voice identity and Emotional style controllable Cross-Lingual Text-to-Speech
by: Gudmalwar, Ashishkumar, et al.
Published: (2024)

EmoReg: Directional Latent Vector Modeling for Emotional Intensity Regularization in Diffusion-based Voice Conversion
by: Gudmalwar, Ashishkumar, et al.
Published: (2024)

REWIND: Speech Time Reversal for Enhancing Speaker Representations in Diffusion-based Voice Conversion
by: Biyani, Ishan D., et al.
Published: (2025)

Isometric Neural Machine Translation using Phoneme Count Ratio Reward-based Reinforcement Learning
by: Mhaskar, Shivam Ratnakant, et al.
Published: (2024)

Attention Is Not Always the Answer: Optimizing Voice Activity Detection with Simple Feature Fusion
by: Tripathi, Kumud, et al.
Published: (2025)

Enhancing Whisper's Accuracy and Speed for Indian Languages through Prompt-Tuning and Tokenization
by: Tripathi, Kumud, et al.
Published: (2024)

Cross-Modal Bottleneck Fusion For Noise Robust Audio-Visual Speech Recognition
by: Ok, Seaone, et al.
Published: (2026)

Joint Multi-scale Cross-lingual Speaking Style Transfer with Bidirectional Attention Mechanism for Automatic Dubbing
by: Li, Jingbei, et al.
Published: (2023)

Windowed SummaryMixing: An Efficient Fine-Tuning of Self-Supervised Learning Models for Low-resource Speech Recognition
by: Menon, Aditya Srinivas, et al.
Published: (2026)

Dub-S2ST: Textless Speech-to-Speech Translation for Seamless Dubbing
by: Choi, Jeongsoo, et al.
Published: (2025)

Improving Audio-Text Retrieval via Hierarchical Cross-Modal Interaction and Auxiliary Captions
by: Xin, Yifei, et al.
Published: (2023)

VoiceCraft-Dub: Automated Video Dubbing with Neural Codec Language Models
by: Sung-Bin, Kim, et al.
Published: (2025)

Audio-Image Cross-Modal Retrieval with Onomatopoeic Images
by: Imoto, Keisuke, et al.
Published: (2026)

EmoDubber: Towards High Quality and Emotion Controllable Movie Dubbing
by: Cong, Gaoxiang, et al.
Published: (2024)

PS-TTS: Phonetic Synchronization in Text-to-Speech for Achieving Natural Automated Dubbing
by: Hong, Changi, et al.
Published: (2026)

AlignFormer: Modality Matching Can Achieve Better Zero-shot Instruction-Following Speech-LLM
by: Fan, Ruchao, et al.
Published: (2024)

DualDub: Video-to-Soundtrack Generation via Joint Speech and Background Audio Synthesis
by: Tian, Wenjie, et al.
Published: (2025)

Musical Source Separation Bake-Off: Comparing Objective Metrics with Human Perception
by: Jaffe, Noah, et al.
Published: (2025)

HD-PPT: Hierarchical Decoding of Content- and Prompt-Preference Tokens for Instruction-based TTS
by: Nie, Sihang, et al.
Published: (2025)

Emotion Recognition in Multi-Speaker Conversations through Speaker Identification, Knowledge Distillation, and Hierarchical Fusion
by: Li, Xiao, et al.
Published: (2025)

Hierarchical Emotion Prediction and Control in Text-to-Speech Synthesis
by: Inoue, Sho, et al.
Published: (2024)

Training Strategies for Modality Dropout Resilient Multi-Modal Target Speaker Extraction
by: Korse, Srikanth, et al.
Published: (2025)

Towards Expressive Video Dubbing with Multiscale Multimodal Context Interaction
by: Zhao, Yuan, et al.
Published: (2024)

Combining Masked Language Modeling and Cross-Modal Contrastive Learning for Prosody-Aware TTS
by: Borodin, Kirill, et al.
Published: (2026)

PoolingVQ: A VQVAE Variant for Reducing Audio Redundancy and Boosting Multi-Modal Fusion in Music Emotion Analysis
by: Zou, Dinghao, et al.
Published: (2025)

Audio-text Retrieval with Transformer-based Hierarchical Alignment and Disentangled Cross-modal Representation
by: Xin, Yifei, et al.
Published: (2024)

Correlation of Fréchet Audio Distance With Human Perception of Environmental Audio Is Embedding Dependant
by: Tailleur, Modan, et al.
Published: (2024)

Exploring Differences between Human Perception and Model Inference in Audio Event Recognition
by: Tan, Yizhou, et al.
Published: (2024)

Length Aware Speech Translation for Video Dubbing
by: Chadha, Harveen Singh, et al.
Published: (2025)

WhisQ: Cross-Modal Representation Learning for Text-to-Music MOS Prediction
by: Emon, Jakaria Islam, et al.
Published: (2025)

Improving Speech Emotion Recognition Through Cross Modal Attention Alignment and Balanced Stacking Model
by: Ueda, Lucas, et al.
Published: (2025)

Predicting Cognitive Decline: A Multimodal AI Approach to Dementia Screening from Speech
by: Chi, Lei, et al.
Published: (2025)

GAP-URGENet: A Generative-Predictive Fusion Framework for Universal Speech Enhancement
by: Rong, Xiaobin, et al.
Published: (2026)

Robots Have Been Seen and Not Heard: Effects of Consequential Sounds on Human-Perception of Robots
by: Allen, Aimee, et al.
Published: (2024)

Multi-Step Prediction and Control of Hierarchical Emotion Distribution in Text-to-Speech Synthesis
by: Inoue, Sho, et al.
Published: (2025)

Deep Dubbing: End-to-End Auto-Audiobook System with Text-to-Timbre and Context-Aware Instruct-TTS
by: Dai, Ziqi, et al.
Published: (2025)

Leveraging Content and Acoustic Representations for Speech Emotion Recognition
by: Dutta, Soumya, et al.
Published: (2024)

From Continuous to Discrete: Cross-Domain Collaborative General Speech Enhancement via Hierarchical Language Models
by: Mu, Zhaoxi, et al.
Published: (2025)