Cover Image

How Visual Representations Map to Language Feature Space in Multimodal LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Venhoff, Constantin, Khakzar, Ashkan, Joseph, Sonia, Torr, Philip, Nanda, Neel
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Understanding Multimodal Fine-Tuning: Spatial Features
by: Naghashyar, Lachin, et al.
Published: (2026)

Too Late to Recall: Explaining the Two-Hop Problem in Multimodal Knowledge Retrieval
by: Venhoff, Constantin, et al.
Published: (2025)

Latent Guard: a Safety Framework for Text-to-image Generation
by: Liu, Runtao, et al.
Published: (2024)

Minimalist Concept Erasure in Generative Models
by: Zhang, Yang, et al.
Published: (2025)

Learning Visual Prompts for Guiding the Attention of Vision Transformers
by: Rezaei, Razieh, et al.
Published: (2024)

Base Models Know How to Reason, Thinking Models Learn When
by: Venhoff, Constantin, et al.
Published: (2025)

Articulate3D: Zero-Shot Text-Driven 3D Object Posing
by: Deb, Oishi, et al.
Published: (2025)

AlignGuard: Scalable Safety Alignment for Text-to-Image Generation
by: Liu, Runtao, et al.
Published: (2024)

Understanding Reasoning in Thinking Language Models via Steering Vectors
by: Venhoff, Constantin, et al.
Published: (2025)

Towards Interpreting Visual Information Processing in Vision-Language Models
by: Neo, Clement, et al.
Published: (2024)

Hidden in Plain Sight: Evaluating Abstract Shape Recognition in Vision-Language Models
by: Hemmat, Arshia, et al.
Published: (2024)

Rethinking Visual Prompting for Multimodal Large Language Models with External Knowledge
by: Lin, Yuanze, et al.
Published: (2024)

Token Activation Map to Visually Explain Multimodal LLMs
by: Li, Yi, et al.
Published: (2025)

Random Representations Outperform Online Continually Learned Representations
by: Prabhu, Ameya, et al.
Published: (2024)

Sparse CLIP: Co-Optimizing Interpretability and Performance in Contrastive Learning
by: Qin, Chuan, et al.
Published: (2026)

Empowering Tuberculosis Screening with Explainable Self-Supervised Deep Neural Networks
by: Patel, Neel, et al.
Published: (2024)

Mixture of Experts Made Intrinsically Interpretable
by: Yang, Xingyi, et al.
Published: (2025)

Reasoning-Finetuning Repurposes Latent Representations in Base Models
by: Ward, Jake, et al.
Published: (2025)

Predictive Regularization Against Visual Representation Degradation in Multimodal Large Language Models
by: Wang, Enguang, et al.
Published: (2026)

Light Cones For Vision: Simple Causal Priors For Visual Hierarchy
by: Kartik, Manglam, et al.
Published: (2026)

Feature-Space Smoothing: Certified Robustness of Deep Representations
by: Xia, Song, et al.
Published: (2026)

Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoders
by: Lan, Michael, et al.
Published: (2024)

Visual Lexicon: Rich Image Features in Language Space
by: Wang, XuDong, et al.
Published: (2024)

Can Multimodal Large Language Models Truly Perform Multimodal In-Context Learning?
by: Chen, Shuo, et al.
Published: (2023)

Learnable Sparsity for Vision Generative Models
by: Zhang, Yang, et al.
Published: (2024)

VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models
by: Han, Junlin, et al.
Published: (2024)

CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor
by: Sun, Shuyang, et al.
Published: (2023)

On the Out-of-Distribution Generalization of Reasoning in Multimodal LLMs for Simple Visual Planning Tasks
by: Neuhaus, Yannic, et al.
Published: (2026)

Efficient Vocabulary-Free Fine-Grained Visual Recognition in the Age of Multimodal LLMs
by: Kuchibhotla, Hari Chandana, et al.
Published: (2025)

Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
by: Zhu, Lianghui, et al.
Published: (2024)

Imagine while Reasoning in Space: Multimodal Visualization-of-Thought
by: Li, Chengzu, et al.
Published: (2025)

T-MARS: Improving Visual Representations by Circumventing Text Feature Learning
by: Maini, Pratyush, et al.
Published: (2023)

Multimodal Fusion Strategies for Mapping Biophysical Landscape Features
by: Gordon, Lucia, et al.
Published: (2024)

Fine-tuning can cripple your foundation model; preserving features may be the solution
by: Mukhoti, Jishnu, et al.
Published: (2023)

Visual Prompting in Multimodal Large Language Models: A Survey
by: Wu, Junda, et al.
Published: (2024)

Introducing Visual Perception Token into Multimodal Large Language Model
by: Yu, Runpeng, et al.
Published: (2025)

Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training
by: Han, Junlin, et al.
Published: (2025)

RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patching
by: Jafari, Farnoush Rezaei, et al.
Published: (2025)

Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models
by: Berasi, Davide, et al.
Published: (2025)

Improving Adversarial Transferability via Model Alignment
by: Ma, Avery, et al.
Published: (2023)