Cover Image

Towards Interpretable and Efficient Attention: Compressing All by Contracting a Few

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wen, Qishuai, Huang, Zhiyuan, Li, Chun-Guang
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mixture-of-Top-k Attention: Efficient Attention via Scalable Fast Weights
by: Wen, Qishuai, et al.
Published: (2026)

Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspective
by: Wen, Qishuai, et al.
Published: (2024)

Exploring a Principled Framework for Deep Subspace Clustering
by: Meng, Xianghan, et al.
Published: (2025)

EDiT: Efficient Diffusion Transformers with Linear Compressed Attention
by: Becker, Philipp, et al.
Published: (2025)

COMCAT: Towards Efficient Compression and Customization of Attention-Based Vision Models
by: Xiao, Jinqi, et al.
Published: (2023)

HEP-NAS: Towards Efficient Few-shot Neural Architecture Search via Hierarchical Edge Partitioning
by: Li, Jianfeng, et al.
Published: (2024)

Towards a Transparent and Interpretable AI Model for Medical Image Classifications
by: Wen, Binbin, et al.
Published: (2025)

Interpretable Few-shot Learning with Online Attribute Selection
by: Zarei, Mohammad Reza, et al.
Published: (2022)

Exploring "Many in Few" and "Few in Many" Properties in Long-Tailed, Highly-Imbalanced IC Defect Classification
by: Shao, Hao-Chiang, et al.
Published: (2025)

Learning to Compress Contexts for Efficient Knowledge-based Visual Question Answering
by: Weng, Weixi, et al.
Published: (2024)

GD-FPS: Growth-Driven Feedforward Parameter Selection for Efficient Fine-Tuning
by: Yang, Kenneth, et al.
Published: (2025)

LoFT: Parameter-Efficient Fine-Tuning for Long-tailed Semi-Supervised Learning in Open-World Scenarios
by: Huang, Zhiyuan, et al.
Published: (2025)

The Missing U for Efficient Diffusion Models
by: Calvo-Ordonez, Sergio, et al.
Published: (2023)

Verbalized Representation Learning for Interpretable Few-Shot Generalization
by: Yang, Cheng-Fu, et al.
Published: (2024)

SVGen: Interpretable Vector Graphics Generation with Large Language Models
by: Wang, Feiyu, et al.
Published: (2025)

Towards Lifelong Few-Shot Customization of Text-to-Image Diffusion
by: Song, Nan, et al.
Published: (2024)

Towards Interpretable Attention Networks for Cervical Cancer Analysis
by: Wang, Ruiqi, et al.
Published: (2021)

ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features
by: Helbling, Alec, et al.
Published: (2025)

Embedding Compression for Efficient Re-Identification
by: McDermott, Luke
Published: (2024)

Communication Efficient Split Learning of ViTs with Attention-based Double Compression
by: Alvetreti, Federico, et al.
Published: (2025)

MonarchRT: Efficient Attention for Real-Time Video Generation
by: Agarwal, Krish, et al.
Published: (2026)

Text Embedding is Not All You Need: Attention Control for Text-to-Image Semantic Alignment with Text Self-Attention Maps
by: Kim, Jeeyung, et al.
Published: (2024)

Cross-Modal Few-Shot Learning: a Generative Transfer Learning Framework
by: Yang, Zhengwei, et al.
Published: (2024)

STAR: Stage-Wise Attention-Guided Token Reduction for Efficient Large Vision-Language Models Inference
by: Guo, Yichen, et al.
Published: (2025)

Few-Shot Class Incremental Learning with Attention-Aware Self-Adaptive Prompt
by: Liu, Chenxi, et al.
Published: (2024)

Rethinking JEPA: Compute-Efficient Video SSL with Frozen Teachers
by: Li, Xianhang, et al.
Published: (2025)

Domain-Enhanced Dual-Branch Model for Efficient and Interpretable Accident Anticipation
by: Guan, Yanchen, et al.
Published: (2025)

Reinforced Attention Learning
by: Li, Bangzheng, et al.
Published: (2026)

Less is More: Efficient Black-box Attribution via Minimal Interpretable Subset Selection
by: Chen, Ruoyu, et al.
Published: (2025)

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?
by: Yu, Yaodong, et al.
Published: (2023)

Saliency Methods are Encoders: Analysing Logical Relations Towards Interpretation
by: Schwenke, Leonid, et al.
Published: (2024)

Perturbation on Feature Coalition: Towards Interpretable Deep Neural Networks
by: Hu, Xuran, et al.
Published: (2024)

Towards Interpreting Visual Information Processing in Vision-Language Models
by: Neo, Clement, et al.
Published: (2024)

ELF-UA: Efficient Label-Free User Adaptation in Gaze Estimation
by: Wu, Yong, et al.
Published: (2024)

Ultra-Efficient Decoding for End-to-End Neural Compression and Reconstruction
by: Rogers, Ethan G., et al.
Published: (2025)

Compressing Model with Few Class-Imbalance Samples: An Out-of-Distribution Expedition
by: Wu, Tian-Shuang, et al.
Published: (2025)

SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezing
by: Li, Sheng, et al.
Published: (2024)

Attention Guided Alignment in Efficient Vision-Language Models
by: Mahajan, Shweta, et al.
Published: (2025)

DataDAM: Efficient Dataset Distillation with Attention Matching
by: Sajedi, Ahmad, et al.
Published: (2023)

Synthesizer Based Efficient Self-Attention for Vision Tasks
by: Zhu, Guangyang, et al.
Published: (2022)