Cover Image

Improving Multimodal Learning with Multi-Loss Gradient Modulation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kontras, Konstantinos, Chatzichristos, Christos, Blaschko, Matthew, De Vos, Maarten
Format:	Preprint
Published:	2024
Subjects:	Multimedia Computer Vision and Pattern Recognition Machine Learning Sound Audio and Speech Processing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flows
by: Li, Shufan, et al.
Published: (2024)

SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion Recognition
by: Cheng, Zebang, et al.
Published: (2024)

MCDubber: Multimodal Context-Aware Expressive Video Dubbing
by: Zhao, Yuan, et al.
Published: (2024)

Design and Development of Laughter Recognition System Based on Multimodal Fusion and Deep Learning
by: Zhao, Fuzheng, et al.
Published: (2024)

Visual and audio scene classification for detecting discrepancies in video: a baseline method and experimental protocol
by: Apostolidis, Konstantinos, et al.
Published: (2024)

Multimodal Music Generation with Explicit Bridges and Retrieval Augmentation
by: Wang, Baisen, et al.
Published: (2024)

Video-Guided Foley Sound Generation with Multimodal Controls
by: Chen, Ziyang, et al.
Published: (2024)

Digit Recognition using Multimodal Spiking Neural Networks
by: Bjorndahl, William, et al.
Published: (2024)

VAInpaint: Zero-Shot Video-Audio inpainting framework with LLMs-driven Module
by: Wu, Kam Man, et al.
Published: (2025)

Pilot-guided Multimodal Semantic Communication for Audio-Visual Event Localization
by: Yu, Fei, et al.
Published: (2024)

MMTrail: A Multimodal Trailer Video Dataset with Language and Music Descriptions
by: Chi, Xiaowei, et al.
Published: (2024)

Music Audio-Visual Question Answering Requires Specialized Multimodal Designs
by: You, Wenhao, et al.
Published: (2025)

Temporal Working Memory: Query-Guided Segment Refinement for Enhanced Multimodal Understanding
by: Diao, Xingjian, et al.
Published: (2025)

Adaptive Audio-Visual Speech Recognition via Matryoshka-Based Multimodal LLMs
by: Cappellazzo, Umberto, et al.
Published: (2025)

Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment
by: Hong, Joanna, et al.
Published: (2025)

FoleyGRAM: Video-to-Audio Generation with GRAM-Aligned Multimodal Encoders
by: Gramaccioni, Riccardo Fosco, et al.
Published: (2025)

M&M: Multimodal-Multitask Model Integrating Audiovisual Cues in Cognitive Load Assessment
by: Nguyen-Phuoc, Long, et al.
Published: (2024)

Mechanisms of Multimodal Synchronization: Insights from Decoder-Based Video-Text-to-Speech Synthesis
by: Gupta, Akshita, et al.
Published: (2024)

AMuSE: Adaptive Multimodal Analysis for Speaker Emotion Recognition in Group Conversations
by: Devulapally, Naresh Kumar, et al.
Published: (2024)

AGAV-Rater: Adapting Large Multimodal Model for AI-Generated Audio-Visual Quality Assessment
by: Cao, Yuqin, et al.
Published: (2025)

Multi-scale Multi-instance Visual Sound Localization and Segmentation
by: Mo, Shentong, et al.
Published: (2024)

AudioGen-Omni: A Unified Multimodal Diffusion Transformer for Video-Synchronized Audio, Speech, and Song Generation
by: Wang, Le, et al.
Published: (2025)

MMS-LLaMA: Efficient LLM-based Audio-Visual Speech Recognition with Minimal Multimodal Speech Tokens
by: Yeo, Jeong Hun, et al.
Published: (2025)

CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment
by: Araujo, Edson, et al.
Published: (2025)

SoundLoc3D: Invisible 3D Sound Source Localization and Classification Using a Multimodal RGB-D Acoustic Camera
by: He, Yuhang, et al.
Published: (2024)

Draw an Audio: Leveraging Multi-Instruction for Video-to-Audio Synthesis
by: Yang, Qi, et al.
Published: (2024)

AutoMV: An Automatic Multi-Agent System for Music Video Generation
by: Tang, Xiaoxuan, et al.
Published: (2025)

UniAV: Unified Audio-Visual Perception for Multi-Task Video Event Localization
by: Geng, Tiantian, et al.
Published: (2024)

AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models
by: Tseng, Yuan, et al.
Published: (2023)

CMMD: Contrastive Multi-Modal Diffusion for Video-Audio Conditional Modeling
by: Yang, Ruihan, et al.
Published: (2023)

Tri-Ergon: Fine-grained Video-to-Audio Generation with Multi-modal Conditions and LUFS Control
by: Li, Bingliang, et al.
Published: (2024)

YingSound: Video-Guided Sound Effects Generation with Multi-modal Chain-of-Thought Controls
by: Chen, Zihao, et al.
Published: (2024)

Audio-Vision Contrastive Learning for Phonological Class Recognition
by: Liu, Daiqi, et al.
Published: (2025)

Learning Musical Representations for Music Performance Question Answering
by: Diao, Xingjian, et al.
Published: (2025)

MMDisCo: Multi-Modal Discriminator-Guided Cooperative Diffusion for Joint Audio and Video Generation
by: Hayakawa, Akio, et al.
Published: (2024)

Sequential Contrastive Audio-Visual Learning
by: Tsiamas, Ioannis, et al.
Published: (2024)

MoMuSE: Momentum Multi-modal Target Speaker Extraction for Real-time Scenarios with Impaired Visual Cues
by: Li, Junjie, et al.
Published: (2024)

Learning Self-Supervised Audio-Visual Representations for Sound Recommendations
by: Krishnamurthy, Sudha
Published: (2024)

Learning Sparsity for Effective and Efficient Music Performance Question Answering
by: Diao, Xingjian, et al.
Published: (2025)

DETECLAP: Enhancing Audio-Visual Representation Learning with Object Information
by: Nakada, Shota, et al.
Published: (2024)