Cover Image

Explaining Caption-Image Interactions in CLIP Models with Second-Order Attributions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Möller, Lucas, Tilli, Pascal, Vu, Ngoc Thang, Padó, Sebastian
Format:	Preprint
Published:	2024
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

HNC: Leveraging Hard Negative Captions towards Models with Fine-Grained Visual-Linguistic Comprehension Capabilities
by: Dönmez, Esra, et al.
Published: (2026)

Thought Flow Nets: From Single Predictions to Trains of Model Thought
by: Schuff, Hendrik, et al.
Published: (2021)

Updating CLIP to Prefer Descriptions Over Captions
by: Zur, Amir, et al.
Published: (2024)

Revisiting Image Captioning Training Paradigm via Direct CLIP-based Optimization
by: Moratelli, Nicholas, et al.
Published: (2024)

Text-only Synthesis for Image Captioning
by: Zhou, Qing, et al.
Published: (2024)

The Role of Data Curation in Image Captioning
by: Li, Wenyan, et al.
Published: (2023)

InterCLIP-MEP: Interactive CLIP and Memory-Enhanced Predictor for Multi-modal Sarcasm Detection
by: Chen, Junjie, et al.
Published: (2024)

Caption First, VQA Second: Knowledge Density, Not Task Format, Drives Multimodal Scaling
by: Zou, Hongjian, et al.
Published: (2026)

The Power of Many: Multi-Agent Multimodal Models for Cultural Image Captioning
by: Bai, Longju, et al.
Published: (2024)

Does CLIP Bind Concepts? Probing Compositionality in Large Image Models
by: Lewis, Martha, et al.
Published: (2022)

CAPEEN: Image Captioning with Early Exits and Knowledge Distillation
by: Bajpai, Divya Jyoti, et al.
Published: (2024)

CIC: A Framework for Culturally-Aware Image Captioning
by: Yun, Youngsik, et al.
Published: (2024)

AC-Lite : A Lightweight Image Captioning Model for Low-Resource Assamese Language
by: Choudhury, Pankaj, et al.
Published: (2025)

Artwork Interpretation with Vision Language Models: A Case Study on Emotions and Emotion Symbols
by: Padó, Sebastian, et al.
Published: (2025)

Contrastive Integrated Gradients: A Feature Attribution-Based Method for Explaining Whole Slide Image Classification
by: Vu, Anh Mai, et al.
Published: (2025)

EXPERT: An Explainable Image Captioning Evaluation Metric with Structured Explanations
by: Kim, Hyunjong, et al.
Published: (2025)

ComCLIP: Training-Free Compositional Image and Text Matching
by: Jiang, Kenan, et al.
Published: (2022)

Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysis
by: Bucciarelli, Davide, et al.
Published: (2024)

Image Captioning Evaluation in the Age of Multimodal LLMs: Challenges and Future Perspectives
by: Sarto, Sara, et al.
Published: (2025)

DENEB: A Hallucination-Robust Automatic Evaluation Metric for Image Captioning
by: Matsuda, Kazuki, et al.
Published: (2024)

Polos: Multimodal Metric Learning from Human Feedback for Image Captioning
by: Wada, Yuiga, et al.
Published: (2024)

FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model
by: Lee, Yebin, et al.
Published: (2024)

Intrinsic Subgraph Generation for Interpretable Graph based Visual Question Answering
by: Tilli, Pascal, et al.
Published: (2024)

Discrete Subgraph Sampling for Interpretable Graph based Visual Question Answering
by: Tilli, Pascal, et al.
Published: (2024)

Towards Retrieval-Augmented Architectures for Image Captioning
by: Sarto, Sara, et al.
Published: (2024)

CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning
by: Xing, Long, et al.
Published: (2025)

VELA: An LLM-Hybrid-as-a-Judge Approach for Evaluating Long Image Captions
by: Matsuda, Kazuki, et al.
Published: (2025)

Structured Captions Improve Prompt Adherence in Text-to-Image Models (Re-LAION-Caption 19M)
by: Merchant, Nicholas, et al.
Published: (2025)

WsiCaption: Multiple Instance Generation of Pathology Reports for Gigapixel Whole-Slide Images
by: Chen, Pingyi, et al.
Published: (2023)

Imagine How To Change: Explicit Procedure Modeling for Change Captioning
by: Sun, Jiayang, et al.
Published: (2026)

LatteCLIP: Unsupervised CLIP Fine-Tuning via LMM-Synthetic Texts
by: Cao, Anh-Quan, et al.
Published: (2024)

Unconstrained Open Vocabulary Image Classification: Zero-Shot Transfer from Text to Image via CLIP Inversion
by: Allgeuer, Philipp, et al.
Published: (2024)

Explaining Black-box Model Predictions via Two-level Nested Feature Attributions with Consistency Property
by: Yoshikawa, Yuya, et al.
Published: (2024)

LAViTeR: Learning Aligned Visual and Textual Representations Assisted by Image and Caption Generation
by: Hashemi, Mohammad Abuzar, et al.
Published: (2021)

Decoding fMRI Data into Captions using Prefix Language Modeling
by: Shen, Vyacheslav, et al.
Published: (2025)

SynC: Synthetic Image Caption Dataset Refinement with One-to-many Mapping for Zero-shot Image Captioning
by: Kim, Si-Woo, et al.
Published: (2025)

Parrot Captions Teach CLIP to Spot Text
by: Lin, Yiqi, et al.
Published: (2023)

Unveiling the Invisible: Captioning Videos with Metaphors
by: Kalarani, Abisek Rajakumar, et al.
Published: (2024)

G-VEval: A Versatile Metric for Evaluating Image and Video Captions Using GPT-4o
by: Tong, Tony Cheng, et al.
Published: (2024)

BRIDGE: Bridging Gaps in Image Captioning Evaluation with Stronger Visual Cues
by: Sarto, Sara, et al.
Published: (2024)