Cover Image

Vision Transformers are Circulant Attention Learners

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Han, Dongchen, Li, Tianyu, Wang, Ziyi, Huang, Gao
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Demystify Mamba in Vision: A Linear Attention Perspective
by: Han, Dongchen, et al.
Published: (2024)

Linear-Time Global Visual Modeling without Explicit Attention
by: He, Ruize, et al.
Published: (2026)

Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials
by: Pu, Yifan, et al.
Published: (2025)

ViT$^3$: Unlocking Test-Time Training in Vision
by: Han, Dongchen, et al.
Published: (2025)

Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediators
by: Pu, Yifan, et al.
Published: (2024)

Agent Attention: On the Integration of Softmax and Linear Attention
by: Han, Dongchen, et al.
Published: (2023)

Bridging the Divide: Reconsidering Softmax and Linear Attention
by: Han, Dongchen, et al.
Published: (2024)

Representative Attention For Vision Transformers
by: Li, Yuntong, et al.
Published: (2026)

S2AFormer: Strip Self-Attention for Efficient Vision Transformer
by: Xu, Guoan, et al.
Published: (2025)

Multi-Attribute Vision Transformers are Efficient and Robust Learners
by: Gani, Hanan, et al.
Published: (2024)

GSVA: Generalized Segmentation via Multimodal Large Language Models
by: Xia, Zhuofan, et al.
Published: (2023)

Structured Initialization for Attention in Vision Transformers
by: Zheng, Jianqiao, et al.
Published: (2024)

Vision Transformers with Hierarchical Attention
by: Liu, Yun, et al.
Published: (2021)

Lightweight Vision Transformer with Window and Spatial Attention for Food Image Classification
by: Gao, Xinle, et al.
Published: (2025)

Self-Supervised Vision Transformers Are Efficient Segmentation Learners for Imperfect Labels
by: Lee, Seungho, et al.
Published: (2024)

Pre-trained Vision and Language Transformers Are Few-Shot Incremental Learners
by: Park, Keon-Hee, et al.
Published: (2024)

Prototypical Transformer as Unified Motion Learners
by: Han, Cheng, et al.
Published: (2024)

Step by Step Network
by: Han, Dongchen, et al.
Published: (2025)

Learning with Unmasked Tokens Drives Stronger Vision Learners
by: Kim, Taekyung, et al.
Published: (2023)

Multi-manifold Attention for Vision Transformers
by: Konstantinidis, Dimitrios, et al.
Published: (2022)

HAViT: Historical Attention Vision Transformer
by: Banik, Swarnendu, et al.
Published: (2026)

Group Diffusion Transformers are Unsupervised Multitask Learners
by: Huang, Lianghua, et al.
Published: (2024)

Contrastive Masked Autoencoders are Stronger Vision Learners
by: Huang, Zhicheng, et al.
Published: (2022)

Video Diffusion Transformers are In-Context Learners
by: Fei, Zhengcong, et al.
Published: (2024)

Polyline Path Masked Attention for Vision Transformer
by: Zhao, Zhongchen, et al.
Published: (2025)

Artifacts and Attention Sinks: Structured Approximations for Efficient Vision Transformers
by: Lu, Andrew, et al.
Published: (2025)

EViT: An Eagle Vision Transformer with Bi-Fovea Self-Attention
by: Shi, Yulong, et al.
Published: (2023)

BinaryAttention: One-Bit QK-Attention for Vision and Diffusion Transformers
by: Xiao, Chaodong, et al.
Published: (2026)

ViT-AdaLA: Adapting Vision Transformers with Linear Attention
by: Li, Yifan, et al.
Published: (2026)

Attn-Adapter: Attention Is All You Need for Online Few-shot Learner of Vision-Language Model
by: Bui, Phuoc-Nguyen, et al.
Published: (2025)

Improving Vision Transformers by Overlapping Heads in Multi-Head Self-Attention
by: Zhang, Tianxiao, et al.
Published: (2024)

Multiple-Exit Tuning: Towards Inference-Efficient Adaptation for Vision Transformer
by: Liu, Zheng, et al.
Published: (2024)

Learning Visual Prompts for Guiding the Attention of Vision Transformers
by: Rezaei, Razieh, et al.
Published: (2024)

Decision-Aware Attention Propagation for Vision Transformer Explainability
by: Jo, Sehyeong, et al.
Published: (2026)

PADRe: A Unifying Polynomial Attention Drop-in Replacement for Efficient Vision Transformer
by: Letourneau, Pierre-David, et al.
Published: (2024)

Collaborative Low-Rank Adaptation for Pre-Trained Vision Transformers
by: Liu, Zheng, et al.
Published: (2025)

Spiking Vision Transformer with Saccadic Attention
by: Wang, Shuai, et al.
Published: (2025)

Simulate, Refocus and Ensemble: An Attention-Refocusing Scheme for Domain Generalization
by: Wang, Ziyi, et al.
Published: (2025)

There is More to Attention: Statistical Filtering Enhances Explanations in Vision Transformers
by: Ayyar, Meghna P, et al.
Published: (2025)

Calibration Attention: Learning Reliability-Aware Representations for Vision Transformers
by: Liang, Wenhao, et al.
Published: (2025)