Cover Image

Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	You, Haoran, Xiong, Yunyang, Dai, Xiaoliang, Wu, Bichen, Zhang, Peizhao, Fan, Haoqi, Vajda, Peter, Lin, Yingyan Celine
Format:	Preprint
Published:	2022
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

INGeo: Accelerating Instant Neural Scene Reconstruction with Noisy Geometry Priors
by: Li, Chaojian, et al.
Published: (2022)

ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformer
by: You, Haoran, et al.
Published: (2023)

FBNetV5: Neural Architecture Search for Multiple Tasks in One Run
by: Wu, Bichen, et al.
Published: (2021)

ViT-AdaLA: Adapting Vision Transformers with Linear Attention
by: Li, Yifan, et al.
Published: (2026)

MixRT: Mixed Neural Representations For Real-Time NeRF Rendering
by: Li, Chaojian, et al.
Published: (2023)

When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
by: You, Haoran, et al.
Published: (2024)

ASCENT-ViT: Attention-based Scale-aware Concept Learning Framework for Enhanced Alignment in Vision Transformers
by: Sinha, Sanchit, et al.
Published: (2025)

Alias-Free ViT: Fractional Shift Invariance via Linear Attention
by: Michaeli, Hagay, et al.
Published: (2025)

Communication Efficient Split Learning of ViTs with Attention-based Double Compression
by: Alvetreti, Federico, et al.
Published: (2025)

ODE-ViT: Plug & Play Attention Layer from the Generalization of the ViT as an Ordinary Differential Equation
by: Riera, Carlos Boned, et al.
Published: (2025)

ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design
by: You, Haoran, et al.
Published: (2022)

PEANO-ViT: Power-Efficient Approximations of Non-Linearities in Vision Transformers
by: Sadeghi, Mohammad Erfan, et al.
Published: (2024)

SAG-ViT: A Scale-Aware, High-Fidelity Patching Approach with Graph Attention for Vision Transformers
by: Venkatraman, Shravan, et al.
Published: (2024)

Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation
by: Zhao, Wangbo, et al.
Published: (2024)

Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence
by: Fu, Yonggan, et al.
Published: (2023)

ViT-5: Vision Transformers for The Mid-2020s
by: Wang, Feng, et al.
Published: (2026)

FCL-ViT: Task-Aware Attention Tuning for Continual Learning
by: Kaimakamidis, Anestis, et al.
Published: (2024)

Early-Bird GCNs: Graph-Network Co-Optimization Towards More Efficient GCN Training and Inference via Drawing Early-Bird Lottery Tickets
by: You, Haoran, et al.
Published: (2021)

Language-Unlocked ViT (LUViT): Empowering Self-Supervised Vision Transformers with LLMs
by: Kuzucu, Selim, et al.
Published: (2025)

ViT-Linearizer: Distilling Quadratic Knowledge into Linear-Time Vision Models
by: Wei, Guoyizhe, et al.
Published: (2025)

ViT-Explainer: An Interactive Walkthrough of the Vision Transformer Pipeline
by: Hernandez, Juan Manuel, et al.
Published: (2026)

ACC-ViT : Atrous Convolution's Comeback in Vision Transformers
by: Ibtehaz, Nabil, et al.
Published: (2024)

EA-ViT: Efficient Adaptation for Elastic Vision Transformer
by: Zhu, Chen, et al.
Published: (2025)

ViT-LCA: A Neuromorphic Approach for Vision Transformers
by: Takaghaj, Sanaz Mahmoodi
Published: (2024)

ViT Registers and Fractal ViT
by: Chou, Jason Chuan-Chih, et al.
Published: (2026)

Self-Attention And Beyond the Infinite: Towards Linear Transformers with Infinite Self-Attention
by: Roffo, Giorgio, et al.
Published: (2026)

DeS3: Adaptive Attention-driven Self and Soft Shadow Removal using ViT Similarity
by: Jin, Yeying, et al.
Published: (2022)

CAS-ViT: Convolutional Additive Self-attention Vision Transformers for Efficient Mobile Applications
by: Zhang, Tianfang, et al.
Published: (2024)

FasterViT: Fast Vision Transformers with Hierarchical Attention
by: Hatamizadeh, Ali, et al.
Published: (2023)

HIRI-ViT: Scaling Vision Transformer with High Resolution Inputs
by: Yao, Ting, et al.
Published: (2024)

IML-ViT: Benchmarking Image Manipulation Localization by Vision Transformer
by: Ma, Xiaochen, et al.
Published: (2023)

Octic Vision Transformers: Quicker ViTs Through Equivariance
by: Nordström, David, et al.
Published: (2025)

VAT: Vision Action Transformer by Unlocking Full Representation of ViT
by: Li, Wenhao, et al.
Published: (2025)

TPC-ViT: Token Propagation Controller for Efficient Vision Transformer
by: Zhu, Wentao
Published: (2024)

An Analysis on Quantizing Diffusion Transformers
by: Yang, Yuewei, et al.
Published: (2024)

MSCViT: A Small-size ViT architecture with Multi-Scale Self-Attention Mechanism for Tiny Datasets
by: Zhang, Bowei, et al.
Published: (2025)

MicroViT: A Vision Transformer with Low Complexity Self Attention for Edge Device
by: Setyawan, Novendra, et al.
Published: (2025)

SAC-ViT: Semantic-Aware Clustering Vision Transformer with Early Exit
by: Hu, Youbing, et al.
Published: (2025)

Prion-ViT: Prions-Inspired Vision Transformers for Temperature prediction with Specklegrams
by: Sebastian, Abhishek, et al.
Published: (2024)

ViT-FIQA: Assessing Face Image Quality using Vision Transformers
by: Atzori, Andrea, et al.
Published: (2025)