Cover Image

SimA: Simple Softmax-free Attention for Vision Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Koohpayegani, Soroush Abbasi, Pirsiavash, Hamed
Format:	Preprint
Published:	2022
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CompGS: Smaller and Faster Gaussian Splatting with Vector Quantization
by: Navaneet, KL, et al.
Published: (2023)

NOLA: Compressing LoRA using Linear Combination of Random Basis
by: Koohpayegani, Soroush Abbasi, et al.
Published: (2023)

GeNIe: Generative Hard Negative Images Through Diffusion
by: Koohpayegani, Soroush Abbasi, et al.
Published: (2023)

Vector-Quantized Soft Label Compression for Dataset Distillation
by: Abbasi, Ali, et al.
Published: (2026)

One Category One Prompt: Dataset Distillation using Diffusion Models
by: Abbasi, Ali, et al.
Published: (2024)

gen2seg: Generative Models Enable Generalizable Instance Segmentation
by: Khangaonkar, Om, et al.
Published: (2025)

Softmax-free Linear Transformers
by: Lu, Jiachen, et al.
Published: (2022)

Rethinking Attention: Polynomial Alternatives to Softmax in Transformers
by: Saratchandran, Hemanth, et al.
Published: (2024)

Agent Attention: On the Integration of Softmax and Linear Attention
by: Han, Dongchen, et al.
Published: (2023)

Multimodal Language Models Cannot Spot Spatial Inconsistencies
by: Khangaonkar, Om, et al.
Published: (2026)

SoLA-Vision: Fine-grained Layer-wise Linear Softmax Hybrid Attention
by: Li, Ruibang, et al.
Published: (2026)

Bridging the Divide: Reconsidering Softmax and Linear Attention
by: Han, Dongchen, et al.
Published: (2024)

SimPLR: A Simple and Plain Transformer for Efficient Object Detection and Segmentation
by: Nguyen, Duy-Kien, et al.
Published: (2023)

Diffusion-Augmented Coreset Expansion for Scalable Dataset Distillation
by: Abbasi, Ali, et al.
Published: (2024)

APLA: A Simple Adaptation Method for Vision Transformers
by: Sorkhei, Moein, et al.
Published: (2025)

Representative Attention For Vision Transformers
by: Li, Yuntong, et al.
Published: (2026)

Vision Transformers with Hierarchical Attention
by: Liu, Yun, et al.
Published: (2021)

Trio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformer
by: Shi, Huihong, et al.
Published: (2024)

SimBase: A Simple Baseline for Temporal Video Grounding
by: Bao, Peijun, et al.
Published: (2024)

Interpretable Image Classification with Adaptive Prototype-based Vision Transformers
by: Ma, Chiyu, et al.
Published: (2024)

Vanilla Group Equivariant Vision Transformer: Simple and Effective
by: Fu, Jiahong, et al.
Published: (2026)

Multi-manifold Attention for Vision Transformers
by: Konstantinidis, Dimitrios, et al.
Published: (2022)

Structured Initialization for Attention in Vision Transformers
by: Zheng, Jianqiao, et al.
Published: (2024)

Vision Transformers are Circulant Attention Learners
by: Han, Dongchen, et al.
Published: (2025)

HAViT: Historical Attention Vision Transformer
by: Banik, Swarnendu, et al.
Published: (2026)

SimToken: A Simple Baseline for Referring Audio-Visual Segmentation
by: Jin, Dian, et al.
Published: (2025)

SimCMF: A Simple Cross-modal Fine-tuning Strategy from Vision Foundation Models to Any Imaging Modality
by: Lei, Chenyang, et al.
Published: (2024)

SimMLM: A Simple Framework for Multi-modal Learning with Missing Modality
by: Li, Sijie, et al.
Published: (2025)

SimInversion: A Simple Framework for Inversion-Based Text-to-Image Editing
by: Qian, Qi, et al.
Published: (2024)

Polyline Path Masked Attention for Vision Transformer
by: Zhao, Zhongchen, et al.
Published: (2025)

BinaryAttention: One-Bit QK-Attention for Vision and Diffusion Transformers
by: Xiao, Chaodong, et al.
Published: (2026)

SimROD: A Simple Baseline for Raw Object Detection with Global and Local Enhancements
by: Xie, Haiyang, et al.
Published: (2025)

LORTSAR: Low-Rank Transformer for Skeleton-based Action Recognition
by: Oraki, Soroush, et al.
Published: (2024)

Learning Visual Prompts for Guiding the Attention of Vision Transformers
by: Rezaei, Razieh, et al.
Published: (2024)

Decision-Aware Attention Propagation for Vision Transformer Explainability
by: Jo, Sehyeong, et al.
Published: (2026)

There is More to Attention: Statistical Filtering Enhances Explanations in Vision Transformers
by: Ayyar, Meghna P, et al.
Published: (2025)

ToSA: Token Selective Attention for Efficient Vision Transformers
by: Singh, Manish Kumar, et al.
Published: (2024)

Artifacts and Attention Sinks: Structured Approximations for Efficient Vision Transformers
by: Lu, Andrew, et al.
Published: (2025)

EViT: An Eagle Vision Transformer with Bi-Fovea Self-Attention
by: Shi, Yulong, et al.
Published: (2023)

ViT-AdaLA: Adapting Vision Transformers with Linear Attention
by: Li, Yifan, et al.
Published: (2026)