Cover Image

If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Esfandiarpoor, Reza, Menghini, Cristina, Bach, Stephen H.
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Computer Vision and Pattern Recognition Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Follow-Up Differential Descriptions: Language Models Resolve Ambiguities for Image Classification
by: Esfandiarpoor, Reza, et al.
Published: (2023)

Enhancing CLIP with CLIP: Exploring Pseudolabeling for Limited-Label Prompt Tuning
by: Menghini, Cristina, et al.
Published: (2023)

Does CLIP Bind Concepts? Probing Compositionality in Large Image Models
by: Lewis, Martha, et al.
Published: (2022)

VideoCLIP-XL: Advancing Long Description Understanding for Video CLIP Models
by: Wang, Jiapeng, et al.
Published: (2024)

Transferring Textual Preferences to Vision-Language Understanding through Model Merging
by: Li, Chen-An, et al.
Published: (2025)

Updating CLIP to Prefer Descriptions Over Captions
by: Zur, Amir, et al.
Published: (2024)

Language Plays a Pivotal Role in the Object-Attribute Compositional Generalization of CLIP
by: Abbasi, Reza, et al.
Published: (2024)

BioCLIP: A Vision Foundation Model for the Tree of Life
by: Stevens, Samuel, et al.
Published: (2023)

When are Lemons Purple? The Concept Association Bias of Vision-Language Models
by: Yamada, Yutaro, et al.
Published: (2022)

Explainable Concept Generation through Vision-Language Preference Learning for Understanding Neural Networks' Internal Representations
by: Taparia, Aditya, et al.
Published: (2024)

Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Models
by: Poppi, Samuele, et al.
Published: (2023)

Aligning Modalities in Vision Large Language Models via Preference Fine-tuning
by: Zhou, Yiyang, et al.
Published: (2024)

Vision-Language Models Create Cross-Modal Task Representations
by: Luo, Grace, et al.
Published: (2024)

TiC-CLIP: Continual Training of CLIP Models
by: Garg, Saurabh, et al.
Published: (2023)

CLIP-Adapter: Better Vision-Language Models with Feature Adapters
by: Gao, Peng, et al.
Published: (2021)

Debiasing Large Vision-Language Models by Ablating Protected Attribute Representations
by: Ratzlaff, Neale, et al.
Published: (2024)

Seeing Through Their Eyes: Evaluating Visual Perspective Taking in Vision Language Models
by: Góral, Gracjan, et al.
Published: (2024)

TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negatives
by: Patel, Maitreya, et al.
Published: (2024)

CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor
by: Sun, Shuyang, et al.
Published: (2023)

Unified Vision-Language Modeling via Concept Space Alignment
by: Qiu, Yifu, et al.
Published: (2026)

HiMo-CLIP: Modeling Semantic Hierarchy and Monotonicity in Vision-Language Alignment
by: Wu, Ruijia, et al.
Published: (2025)

LLM2CLIP: Powerful Language Model Unlocks Richer Cross-Modality Representation
by: Huang, Weiquan, et al.
Published: (2024)

Sim-CLIP: Unsupervised Siamese Adversarial Fine-Tuning for Robust and Semantically-Rich Vision-Language Models
by: Hossain, Md Zarif, et al.
Published: (2024)

Pre-trained Vision-Language Models Learn Discoverable Visual Concepts
by: Zang, Yuan, et al.
Published: (2024)

Vision-Based Natural Language Scene Understanding for Autonomous Driving: An Extended Dataset and a New Model for Traffic Scene Description Generation
by: Zadeh, Danial Sadrian, et al.
Published: (2026)

Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision Encoder
by: Li, Siting, et al.
Published: (2024)

Understanding the Effects of Distractors on Reasoning Vision-Language Models
by: Bae, Jiyun, et al.
Published: (2025)

From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models
by: Bhatia, Mehar, et al.
Published: (2024)

VCM: Vision Concept Modeling Based on Implicit Contrastive Learning with Vision-Language Instruction Fine-Tuning
by: Luo, Run, et al.
Published: (2025)

A Semantic Space is Worth 256 Language Descriptions: Make Stronger Segmentation Models with Descriptive Properties
by: Xiao, Junfei, et al.
Published: (2023)

Enhancing Abnormality Grounding for Vision Language Models with Knowledge Descriptions
by: Li, Jun, et al.
Published: (2025)

BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning
by: Xu, Xiao, et al.
Published: (2022)

Seeing is Believing: Mitigating Hallucination in Large Vision-Language Models via CLIP-Guided Decoding
by: Deng, Ailin, et al.
Published: (2024)

Vision-Language Models Do Not Understand Negation
by: Alhamoud, Kumail, et al.
Published: (2025)

Do Vision and Language Models Share Concepts? A Vector Space Alignment Study
by: Li, Jiaang, et al.
Published: (2023)

Benchmarking Direct Preference Optimization for Medical Large Vision-Language Models
by: Kim, Dain, et al.
Published: (2026)

Do Vision-Language Models Really Understand Visual Language?
by: Hou, Yifan, et al.
Published: (2024)

Exploring Multi-Grained Concept Annotations for Multimodal Large Language Models
by: Xu, Xiao, et al.
Published: (2024)

The Neglected Tails in Vision-Language Models
by: Parashar, Shubham, et al.
Published: (2024)

CLIP-SVD: Efficient and Interpretable Vision-Language Adaptation via Singular Values
by: Koleilat, Taha, et al.
Published: (2025)