Cover Image

Q Cache: Visual Attention is Valuable in Less than Half of Decode Layers for Multimodal Large Language Model

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhuang, Jiedong, Lu, Lu, Dai, Ming, Hu, Rui, Chen, Jian, Liu, Qiang, Hu, Haoji
Format:	Preprint
Published:	2026
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ST$^3$: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming
by: Zhuang, Jiedong, et al.
Published: (2024)

Mitigating Hallucination in Visual-Language Models via Re-Balancing Contrastive Decoding
by: Liang, Xiaoyu, et al.
Published: (2024)

FALIP: Visual Prompt as Foveal Attention Boosts CLIP Zero-Shot Performance
by: Zhuang, Jiedong, et al.
Published: (2024)

Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints
by: Dai, Ming, et al.
Published: (2025)

Training-Free Multi-View Extension of IC-Light for Textual Position-Aware Scene Relighting
by: Ye, Jiangnan, et al.
Published: (2025)

Visual Attention Drifts,but Anchors Hold:Mitigating Hallucination in Multimodal Large Language Models via Cross-Layer Visual Anchors
by: Yang, Chengxu, et al.
Published: (2026)

TeG-DG: Textually Guided Domain Generalization for Face Anti-Spoofing
by: Mu, Lianrui, et al.
Published: (2023)

No Pixel Left Behind: A Detail-Preserving Architecture for Robust High-Resolution AI-Generated Image Detection
by: Mu, Lianrui, et al.
Published: (2025)

EvoCut: Multi-Layer Evolution-Aware Visual Token Compression for Efficient Large Vision-Language Models
by: Lu, Hongyu, et al.
Published: (2026)

PropVG: End-to-End Proposal-Driven Visual Grounding with Multi-Granularity Discrimination
by: Dai, Ming, et al.
Published: (2025)

Dynamic Token Reduction during Generation for Vision Language Models
by: Liang, Xiaoyu, et al.
Published: (2025)

EDTformer: An Efficient Decoder Transformer for Visual Place Recognition
by: Jin, Tong, et al.
Published: (2024)

Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models
by: Zhou, Qiji, et al.
Published: (2024)

D3S2: Diffusion-Guided Dataset Distillation for Semantic Segmentation
by: Zheng, Wenjie, et al.
Published: (2026)

ShortV: Efficient Multimodal Large Language Models by Freezing Visual Tokens in Ineffective Layers
by: Yuan, Qianhao, et al.
Published: (2025)

Predictive Regularization Against Visual Representation Degradation in Multimodal Large Language Models
by: Wang, Enguang, et al.
Published: (2026)

LightVLM: Acceleraing Large Multimodal Models with Pyramid Token Merging and KV Cache Compression
by: Hu, Lianyu, et al.
Published: (2025)

Fork-Merge Decoding: Enhancing Multimodal Understanding in Audio-Visual Large Language Models
by: Jung, Chaeyoung, et al.
Published: (2025)

ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models
by: Tu, Yahan, et al.
Published: (2024)

Less Is More? Selective Visual Attention to High-Importance Regions for Multimodal Radiology Summarization
by: Naznin, Mst. Fahmida Sultana, et al.
Published: (2026)

Period-LLM: Extending the Periodic Capability of Multimodal Large Language Model
by: Zhang, Yuting, et al.
Published: (2025)

TalkFashion: Intelligent Virtual Try-On Assistant Based on Multimodal Large Language Model
by: Hu, Yujie, et al.
Published: (2025)

DUALVISION: RGB-Infrared Multimodal Large Language Models for Robust Visual Reasoning
by: Majeedi, Abrar, et al.
Published: (2026)

MusiXQA: Advancing Visual Music Understanding in Multimodal Large Language Models
by: Chen, Jian, et al.
Published: (2025)

FashionR2R: Texture-preserving Rendered-to-Real Image Translation with Diffusion Models
by: Hu, Rui, et al.
Published: (2024)

Exploring Implicit Visual Misunderstandings in Multimodal Large Language Models through Attention Analysis
by: Wang, Pengfei, et al.
Published: (2025)

Mixture of Decoding: An Attention-Inspired Adaptive Decoding Strategy to Mitigate Hallucinations in Large Vision-Language Models
by: Chen, Xinlong, et al.
Published: (2025)

GAMBIT: A Gamified Jailbreak Framework for Multimodal Large Language Models
by: Hu, Xiangdong, et al.
Published: (2026)

VaLiD: Mitigating the Hallucination of Large Vision Language Models by Visual Layer Fusion Contrastive Decoding
by: Wang, Jiaqi, et al.
Published: (2024)

Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality
by: Zhou, Guanyu, et al.
Published: (2024)

Seeing Is Believing? A Benchmark for Multimodal Large Language Models on Visual Illusions and Anomalies
by: Hou, Wenjin, et al.
Published: (2026)

LaCo: Efficient Layer-wise Compression of Visual Tokens for Multimodal Large Language Models
by: Liu, Juntao, et al.
Published: (2025)

Extracting Visual Facts from Intermediate Layers for Mitigating Hallucinations in Multimodal Large Language Models
by: Zhou, Haoran, et al.
Published: (2025)

Multi-Layer Dense Attention Decoder for Polyp Segmentation
by: Patel, Krushi, et al.
Published: (2024)

GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing
by: Fang, Rongyao, et al.
Published: (2025)

Learning to Decode Against Compositional Hallucination in Video Multimodal Large Language Models
by: Xing, Wenbin, et al.
Published: (2026)

A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models
by: Li, Duo, et al.
Published: (2025)

VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model
by: Zhuang, Xianwei, et al.
Published: (2025)

Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
by: Dong, Yuhao, et al.
Published: (2024)

SOWA: Adapting Hierarchical Frozen Window Self-Attention to Visual-Language Models for Better Anomaly Detection
by: Hu, Zongxiang, et al.
Published: (2024)