Cover Image

SteerVLM: Robust Model Control through Lightweight Activation Steering for Vision Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sivakumar, Anushka, Zhang, Andrew, Hakim, Zaber, Thomas, Chris
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Steering to Say No: Configurable Refusal via Activation Steering in Vision Language Models
by: Yang, Jiaxi, et al.
Published: (2026)

Maximal Matching Matters: Preventing Representation Collapse for Robust Cross-Modal Retrieval
by: Alomari, Hani, et al.
Published: (2025)

Flexible-length Text Infilling for Discrete Diffusion Models
by: Zhang, Andrew, et al.
Published: (2025)

Exploring Attention Mechanisms in Integration of Multi-Modal Information for Sign Language Recognition and Translation
by: Hakim, Zaber Ibn Abdul, et al.
Published: (2023)

Steering Rectified Flow Models in the Vector Field for Controlled Image Generation
by: Patel, Maitreya, et al.
Published: (2024)

VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision
by: Xu, Yi, et al.
Published: (2024)

Reducing Hallucinations in Vision-Language Models via Latent Space Steering
by: Liu, Sheng, et al.
Published: (2024)

SAVeS: Steering Safety Judgments in Vision-Language Models via Semantic Cues
by: Hinojosa, Carlos, et al.
Published: (2026)

Image Inpainting via Tractable Steering of Diffusion Models
by: Liu, Anji, et al.
Published: (2023)

Steering Guidance for Personalized Text-to-Image Diffusion Models
by: Park, Sunghyun, et al.
Published: (2025)

Test-Time Spectrum-Aware Latent Steering for Zero-Shot Generalization in Vision-Language Models
by: Dafnis, Konstantinos M., et al.
Published: (2025)

The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering
by: Li, Zhuowei, et al.
Published: (2025)

GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning
by: Liu, Jiajin, et al.
Published: (2026)

HandsOnVLM: Vision-Language Models for Hand-Object Interaction Prediction
by: Bao, Chen, et al.
Published: (2024)

Steer Away From Mode Collisions: Improving Composition In Diffusion Models
by: Dutta, Debottam, et al.
Published: (2025)

BendVLM: Test-Time Debiasing of Vision-Language Embeddings
by: Gerych, Walter, et al.
Published: (2024)

Beyond ImageNet: Understanding Cross-Dataset Robustness of Lightweight Vision Models
by: Zhang, Weidong, et al.
Published: (2025)

Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Models
by: Gan, Woody Haosheng, et al.
Published: (2025)

FastVLM: Efficient Vision Encoding for Vision Language Models
by: Vasu, Pavan Kumar Anasosalu, et al.
Published: (2024)

A Lightweight Large Vision-language Model for Multimodal Medical Images
by: Alsinglawi, Belal, et al.
Published: (2025)

Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting
by: Zhong, Siru, et al.
Published: (2025)

StressDream: Steering Video World Models for Robust Policy Evaluation and Improvement
by: Seo, Junwon, et al.
Published: (2026)

Learning to Steer: Input-dependent Steering for Multimodal LLMs
by: Parekh, Jayneel, et al.
Published: (2025)

A General Framework for Inference-time Scaling and Steering of Diffusion Models
by: Singhal, Raghav, et al.
Published: (2025)

Steering Sparse Autoencoder Latents to Control Dynamic Head Pruning in Vision Transformers (Student Abstract)
by: Lee, Yousung, et al.
Published: (2026)

Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuning
by: Koleilat, Taha, et al.
Published: (2026)

LAMP: Learning Universal Adversarial Perturbations for Multi-Image Tasks via Pre-trained Models
by: Ishmam, Alvi Md, et al.
Published: (2026)

SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
by: Chen, Boyuan, et al.
Published: (2024)

Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks
by: Valiente, Rodolfo, et al.
Published: (2019)

Debias your Large Multi-Modal Model at Test-Time via Non-Contrastive Visual Attribute Steering
by: Ratzlaff, Neale, et al.
Published: (2024)

Model Steering: Learning with a Reference Model Improves Generalization Bounds and Scaling Laws
by: Wei, Xiyuan, et al.
Published: (2025)

Anatomy-VLM: A Fine-grained Vision-Language Model for Medical Interpretation
by: Gu, Difei, et al.
Published: (2025)

ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area
by: Li, Junxian, et al.
Published: (2024)

ATHENA: Adaptive Test-Time Steering for Improving Count Fidelity in Diffusion Models
by: Sepehri, Mohammad Shahab, et al.
Published: (2026)

Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms
by: Wang, Mengru, et al.
Published: (2025)

Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language Models through Visual Counterfacts
by: Golovanevsky, Michal, et al.
Published: (2025)

Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models
by: Yin, Jianghao, et al.
Published: (2026)

FREE: Fast and Robust Vision Language Models with Early Exits
by: Bajpai, Divya Jyoti, et al.
Published: (2025)

SteeringDiffusion: A Bottlenecked Activation Control Interface for Diffusion Models
by: Wu, Fangzheng, et al.
Published: (2026)

Lightweight Unsupervised Federated Learning with Pretrained Vision Language Model
by: Yan, Hao, et al.
Published: (2024)