Cover Image

Automatic Channel Pruning for Multi-Head Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Eunho, Hwang, Youngbae
Format:	Preprint
Published:	2024
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence Computational Complexity
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts
by: Lee, Eunho, et al.
Published: (2026)

On the Intrinsic Limits of Transformer Image Embeddings in Non-Solvable Spatial Reasoning
by: Lyu, Siyi, et al.
Published: (2026)

Filter Pruning based on Information Capacity and Independence
by: Tang, Xiaolong, et al.
Published: (2023)

PruNeRF: Segment-Centric Dataset Pruning via 3D Spatial Consistency
by: Jung, Yeonsung, et al.
Published: (2024)

fruit-SALAD: A Style Aligned Artwork Dataset to reveal similarity perception in image embeddings
by: Ohm, Tillmann, et al.
Published: (2024)

On Computational Limits of FlowAR Models: Expressivity and Efficiency
by: Cao, Yang, et al.
Published: (2025)

On Computational Limits and Provably Efficient Criteria of Visual Autoregressive Models: A Fine-Grained Complexity Analysis
by: Ke, Yekun, et al.
Published: (2025)

MoH: Multi-Head Attention as Mixture-of-Head Attention
by: Jin, Peng, et al.
Published: (2024)

Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding
by: Kang, Seil, et al.
Published: (2025)

Selective LoRA for Visual Tokens and Attention Heads
by: Luo, Tiange, et al.
Published: (2025)

Efficient Multi-Object Tracking on Edge Devices via Reconstruction-Based Channel Pruning
by: Müller, Jan, et al.
Published: (2024)

Integrating Multimodal Large Language Model Knowledge into Amodal Completion
by: Yun, Heecheol, et al.
Published: (2026)

Model Compression using Progressive Channel Pruning
by: Guo, Jinyang, et al.
Published: (2025)

AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security
by: Liu, Dongrui, et al.
Published: (2026)

Multi-Dimensional Pruning: Joint Channel, Layer and Block Pruning with Latency Constraint
by: Sun, Xinglong, et al.
Published: (2024)

REPrune: Channel Pruning via Kernel Representative Selection
by: Park, Mincheol, et al.
Published: (2024)

MHLA: Restoring Expressivity of Linear Attention via Token-Level Multi-Head
by: Zhang, Kewei, et al.
Published: (2026)

Achieving Fairness Through Channel Pruning for Dermatological Disease Diagnosis
by: Kong, Qingpeng, et al.
Published: (2024)

Preserve or Modify? Context-Aware Evaluation for Balancing Preservation and Modification in Text-Guided Image Editing
by: Kim, Yoonjeon, et al.
Published: (2024)

SNP: Structured Neuron-level Pruning to Preserve Attention Scores
by: Shim, Kyunghwan, et al.
Published: (2024)

Enhancing Layer Attention Efficiency through Pruning Redundant Retrievals
by: Li, Hanze, et al.
Published: (2025)

Object-level Cross-view Geo-localization with Location Enhancement and Multi-Head Cross Attention
by: Huang, Zheyang, et al.
Published: (2025)

Mitigating Vanishing Activations in Deep CapsNets Using Channel Pruning
by: Sahu, Siddharth, et al.
Published: (2024)

MAST: Mask-Guided Attention Mass Allocation for Training-Free Multi-Style Transfer
by: Kang, Dongkyung, et al.
Published: (2026)

Pruning the Paradox: How CLIP's Most Informative Heads Enhance Performance While Amplifying Bias
by: Madasu, Avinash, et al.
Published: (2025)

IWP: Token Pruning as Implicit Weight Pruning in Large Vision Language Models
by: Lee, Dong-Jae, et al.
Published: (2026)

DiTFastAttnV2: Head-wise Attention Compression for Multi-Modality Diffusion Transformers
by: Zhang, Hanling, et al.
Published: (2025)

Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs
by: Zhang, Qizhe, et al.
Published: (2025)

Steering Sparse Autoencoder Latents to Control Dynamic Head Pruning in Vision Transformers (Student Abstract)
by: Lee, Yousung, et al.
Published: (2026)

UniHead: Unifying Multi-Perception for Detection Heads
by: Zhou, Hantao, et al.
Published: (2023)

Rotation-Aligned Key Channel Pruning for Efficient Vision-Language Model Inference
by: Kang, Beomseok, et al.
Published: (2026)

PLPHP: Per-Layer Per-Head Vision Token Pruning for Efficient Large Vision-Language Models
by: Meng, Yu, et al.
Published: (2025)

Generalized Uncertainty-Based Evidential Fusion with Hybrid Multi-Head Attention for Weak-Supervised Temporal Action Localization
by: He, Yuanpeng, et al.
Published: (2024)

Attention in Space: Functional Roles of VLM Heads for Spatial Reasoning
by: Ma, Xueqi, et al.
Published: (2026)

Interpreting Attention Heads for Image-to-Text Information Flow in Large Vision-Language Models
by: Kim, Jinyeong, et al.
Published: (2025)

AdaTP: Attention-Debiased Token Pruning for Video Large Language Models
by: Sun, Fengyuan, et al.
Published: (2025)

WaveNets: Wavelet Channel Attention Networks
by: Salman, Hadi, et al.
Published: (2022)

Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs
by: Zhang, Qizhe, et al.
Published: (2024)

Med-PerSAM: One-Shot Visual Prompt Tuning for Personalized Segment Anything Model in Medical Domain
by: Yoon, Hangyul, et al.
Published: (2024)

Unveiling the Response of Large Vision-Language Models to Visually Absent Tokens
by: Kim, Sohee, et al.
Published: (2025)