Cover Image

Decipher the Modality Gap in Multimodal Contrastive Learning: From Convergent Representations to Pairwise Alignment

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yi, Lingjie, Douady, Raphael, Chen, Chao
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Explaining and Mitigating the Modality Gap in Contrastive Multimodal Learning
by: Yaras, Can, et al.
Published: (2024)

Less is More: Multimodal Region Representation via Pairwise Inter-view Learning
by: Namgung, Min, et al.
Published: (2025)

Enhancing Modality Representation and Alignment for Multimodal Cold-start Active Learning
by: Shen, Meng, et al.
Published: (2024)

Pairwise Calibrated Rewards for Pluralistic Alignment
by: Halpern, Daniel, et al.
Published: (2025)

On the Spectral Geometry of Cross-Modal Representations: A Functional Map Diagnostic for Multimodal Alignment
by: Sarkar, Krisanu
Published: (2026)

Fast and Featureless Node Representation Learning with Partial Pairwise Supervision
by: Chakraborty, Sujan, et al.
Published: (2026)

Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representations
by: Luo, Haozheng, et al.
Published: (2026)

Causal Debiasing Medical Multimodal Representation Learning with Missing Modalities
by: Zhu, Xiaoguang, et al.
Published: (2025)

Multimodal Physiological Signals Representation Learning via Multiscale Contrasting for Depression Recognition
by: Shao, Kai, et al.
Published: (2024)

Understanding the Emergence of Multimodal Representation Alignment
by: Tjandrasuwita, Megan, et al.
Published: (2025)

Quantifying Multimodal Capabilities: Formal Generalization Guarantees in Pairwise Metric Learning
by: Zhou, Richeng, et al.
Published: (2026)

Variance Alignment Score: A Simple But Tough-to-Beat Data Selection Method for Multimodal Contrastive Learning
by: Wang, Yiping, et al.
Published: (2024)

On the Exponential Convergence for Offline RLHF with Pairwise Comparisons
by: Chen, Zhirui, et al.
Published: (2024)

Pairwise Difference Learning for Classification
by: Belaid, Mohamed Karim, et al.
Published: (2024)

Gramian Multimodal Representation Learning and Alignment
by: Cicchetti, Giordano, et al.
Published: (2024)

Beyond Pairwise: Empowering LLM Alignment With Ranked Choice Modeling
by: Tang, Yuxuan, et al.
Published: (2025)

From Alignment to Prediction: A Study of Self-Supervised Learning and Predictive Representation Learning
by: Dutta, Mintu, et al.
Published: (2026)

Preference-based Reinforcement Learning beyond Pairwise Comparisons: Benefits of Multiple Options
by: Lee, Joongkyu, et al.
Published: (2025)

On the Representation of Pairwise Causal Background Knowledge and Its Applications in Causal Inference
by: Fang, Zhuangyan, et al.
Published: (2022)

Contrastive Learning Via Equivariant Representation
by: Song, Sifan, et al.
Published: (2024)

Lightweight Cross-Modal Representation Learning
by: Faye, Bilal, et al.
Published: (2024)

Material Property Prediction with Element Attribute Knowledge Graphs and Multimodal Representation Learning
by: Huang, Chao, et al.
Published: (2024)

Spectral Disentanglement and Enhancement: A Dual-domain Contrastive Framework for Representation Learning
by: Guo, Jinjin, et al.
Published: (2026)

Perfect Alignment May be Poisonous to Graph Contrastive Learning
by: Liu, Jingyu, et al.
Published: (2023)

Multimodal Visual-Tactile Representation Learning through Self-Supervised Contrastive Pre-Training
by: Dave, Vedant, et al.
Published: (2024)

COMET: Concept Space Dissection of the Modality Gap in Audio-Text Multimodal Contrastive Embeddings
by: Zhu, Yonggang, et al.
Published: (2026)

MLEM: Generative and Contrastive Learning as Distinct Modalities for Event Sequences
by: Moskvoretskii, Viktor, et al.
Published: (2024)

Leveraging Superfluous Information in Contrastive Representation Learning
by: Yu, Xuechu
Published: (2024)

From Abstract to Actionable: Pairwise Shapley Values for Explainable AI
by: Xu, Jiaxin, et al.
Published: (2025)

Learning Linear Utility Functions From Pairwise Comparison Queries
by: Ge, Luise, et al.
Published: (2024)

Multimodal Federated Learning with Missing Modality via Prototype Mask and Contrast
by: Bao, Guangyin, et al.
Published: (2023)

Towards a Learning Theory of Representation Alignment
by: Insulla, Francesco, et al.
Published: (2025)

Enhance Exploration in Safe Reinforcement Learning with Contrastive Representation Learning
by: Doan, Duc Kien, et al.
Published: (2025)

Quantifying Modality Contributions via Disentangling Multimodal Representations
by: Amit, Padegal, et al.
Published: (2025)

Representation Learning with Mutual Influence of Modalities for Node Classification in Multi-Modal Heterogeneous Networks
by: Li, Jiafan, et al.
Published: (2025)

Structure-Aware Fusion with Progressive Injection for Multimodal Molecular Representation Learning
by: Jing, Zihao, et al.
Published: (2025)

Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneity
by: Zhang, Sirui, et al.
Published: (2026)

FANoise: Singular Value-Adaptive Noise Modulation for Robust Multimodal Representation Learning
by: Li, Jiaoyang, et al.
Published: (2025)

Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMs
by: Zhang, Dingkun, et al.
Published: (2025)

Robust Multimodal Representation Learning in Healthcare
by: Zhu, Xiaoguang, et al.
Published: (2026)