Cover Image

Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision Encoder

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Siting, Koh, Pang Wei, Du, Simon Shaolei
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Highlighting What Matters: Promptable Embeddings for Attribute-Focused Image Retrieval
by: Li, Siting, et al.
Published: (2025)

Fine-tuning MLLMs Without Forgetting Is Easier Than You Think
by: Li, He, et al.
Published: (2026)

JPEG-LM: LLMs as Image Generators with Canonical Codec Representations
by: Han, Xiaochuang, et al.
Published: (2024)

What Makes CLIP More Robust to Long-Tailed Pre-Training Data? A Controlled Study for Transferable Insights
by: Wen, Xin, et al.
Published: (2024)

ICONS: Influence Consensus for Vision-Language Data Selection
by: Wu, Xindi, et al.
Published: (2024)

BioCLIP: A Vision Foundation Model for the Tree of Life
by: Stevens, Samuel, et al.
Published: (2023)

Injecting Distributional Awareness into MLLMs via Reinforcement Learning for Deep Imbalanced Regression
by: Du, Yao, et al.
Published: (2026)

Towards Efficient Vision-Language Tuning: More Information Density, More Generalizability
by: Hao, Tianxiang, et al.
Published: (2023)

Teach CLIP to Develop a Number Sense for Ordinal Regression
by: Du, Yao, et al.
Published: (2024)

Revisiting Compositionality in Dual-Encoder Vision-Language Models: The Role of Inference
by: Miranda, Imanol, et al.
Published: (2026)

TiC-CLIP: Continual Training of CLIP Models
by: Garg, Saurabh, et al.
Published: (2023)

BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning
by: Xu, Xiao, et al.
Published: (2022)

If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions
by: Esfandiarpoor, Reza, et al.
Published: (2024)

Language Plays a Pivotal Role in the Object-Attribute Compositional Generalization of CLIP
by: Abbasi, Reza, et al.
Published: (2024)

The Instinctive Bias: Spurious Images lead to Illusion in MLLMs
by: Han, Tianyang, et al.
Published: (2024)

Renaissance: Investigating the Pretraining of Vision-Language Encoders
by: Fields, Clayton, et al.
Published: (2024)

When and How Does CLIP Enable Domain and Compositional Generalization?
by: Kempf, Elias, et al.
Published: (2025)

Imperfect Vision Encoders: Efficient and Robust Tuning for Vision-Language Models
by: Panos, Aristeidis, et al.
Published: (2024)

Do Vision and Language Encoders Represent the World Similarly?
by: Maniparambil, Mayug, et al.
Published: (2024)

Frozen Transformers in Language Models Are Effective Visual Encoder Layers
by: Pang, Ziqi, et al.
Published: (2023)

Words That Make Language Models Perceive
by: Wang, Sophie L., et al.
Published: (2025)

Manager: Aggregating Insights from Unimodal Experts in Two-Tower VLMs and MLLMs
by: Xu, Xiao, et al.
Published: (2025)

BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning
by: Gu, Jianyang, et al.
Published: (2025)

Semantic Token Reweighting for Interpretable and Controllable Text Embeddings in CLIP
by: Kim, Eunji, et al.
Published: (2024)

M2-Reasoning: Empowering MLLMs with Unified General and Spatial Reasoning
by: AI, Inclusion, et al.
Published: (2025)

Unifying Specialized Visual Encoders for Video Language Models
by: Chung, Jihoon, et al.
Published: (2025)

D$^{3}$ToM: Decider-Guided Dynamic Token Merging for Accelerating Diffusion MLLMs
by: Chang, Shuochen, et al.
Published: (2025)

IPO: Interpretable Prompt Optimization for Vision-Language Models
by: Du, Yingjun, et al.
Published: (2024)

Exploring CLIP's Dense Knowledge for Weakly Supervised Semantic Segmentation
by: Yang, Zhiwei, et al.
Published: (2025)

An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels
by: Nguyen, Duy-Kien, et al.
Published: (2024)

AMU-Tuning: Effective Logit Bias for CLIP-based Few-shot Learning
by: Tang, Yuwei, et al.
Published: (2024)

DELAN: Dual-Level Alignment for Vision-and-Language Navigation by Cross-Modal Contrastive Learning
by: Du, Mengfei, et al.
Published: (2024)

TLDR: Token-Level Detective Reward Model for Large Vision Language Models
by: Fu, Deqing, et al.
Published: (2024)

MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training
by: Vasu, Pavan Kumar Anasosalu, et al.
Published: (2023)

CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collections
by: Imam, Mohamed Fazli, et al.
Published: (2024)

Using Knowledge Graphs to harvest datasets for efficient CLIP model training
by: Ging, Simon, et al.
Published: (2025)

Exploring the Limits of Zero Shot Vision Language Models for Hate Meme Detection: The Vulnerabilities and their Interpretations
by: Rizwan, Naquee, et al.
Published: (2024)

How Vision-Language Tasks Benefit from Large Pre-trained Models: A Survey
by: Qi, Yayun, et al.
Published: (2024)

Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy
by: Ging, Simon, et al.
Published: (2024)

Sim-CLIP: Unsupervised Siamese Adversarial Fine-Tuning for Robust and Semantically-Rich Vision-Language Models
by: Hossain, Md Zarif, et al.
Published: (2024)