Cover Image

Bridging the Divide: Reconsidering Softmax and Linear Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Han, Dongchen, Pu, Yifan, Xia, Zhuofan, Han, Yizeng, Pan, Xuran, Li, Xiu, Lu, Jiwen, Song, Shiji, Huang, Gao
Format:	Preprint
Published:	2024
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Agent Attention: On the Integration of Softmax and Linear Attention
by: Han, Dongchen, et al.
Published: (2023)

GSVA: Generalized Segmentation via Multimodal Large Language Models
by: Xia, Zhuofan, et al.
Published: (2023)

Demystify Mamba in Vision: A Linear Attention Perspective
by: Han, Dongchen, et al.
Published: (2024)

Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediators
by: Pu, Yifan, et al.
Published: (2024)

Latency-aware Unified Dynamic Networks for Efficient Image Recognition
by: Han, Yizeng, et al.
Published: (2023)

GRA: Detecting Oriented Objects through Group-wise Rotating and Attention
by: Wang, Jiangshan, et al.
Published: (2024)

Linear-Time Global Visual Modeling without Explicit Attention
by: He, Ruize, et al.
Published: (2026)

EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training
by: Wang, Yulin, et al.
Published: (2024)

DyFADet: Dynamic Feature Aggregation for Temporal Action Detection
by: Yang, Le, et al.
Published: (2024)

Vision Transformers are Circulant Attention Learners
by: Han, Dongchen, et al.
Published: (2025)

Step by Step Network
by: Han, Dongchen, et al.
Published: (2025)

Exploring contextual modeling with linear complexity for point cloud segmentation
by: Chng, Yong Xien, et al.
Published: (2024)

Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials
by: Pu, Yifan, et al.
Published: (2025)

Few-Step Distillation for Text-to-Image Generation: A Practical Guide
by: Pu, Yifan, et al.
Published: (2025)

SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning
by: Du, Chaoqun, et al.
Published: (2024)

Emulating Human-like Adaptive Vision for Efficient and Flexible Machine Visual Perception
by: Wang, Yulin, et al.
Published: (2025)

SoLA-Vision: Fine-grained Layer-wise Linear Softmax Hybrid Attention
by: Li, Ruibang, et al.
Published: (2026)

Mask Grounding for Referring Image Segmentation
by: Chng, Yong Xien, et al.
Published: (2023)

LINA: Linear Autoregressive Image Generative Models with Continuous Tokens
by: Wang, Jiahao, et al.
Published: (2026)

Softmax-free Linear Transformers
by: Lu, Jiachen, et al.
Published: (2022)

Adapting Vision-Language Model with Fine-grained Semantics for Open-Vocabulary Segmentation
by: Chng, Yong Xien, et al.
Published: (2024)

EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction
by: Cai, Han, et al.
Published: (2022)

Dynamic Diffusion Transformer
by: Zhao, Wangbo, et al.
Published: (2024)

Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation
by: Zhao, Wangbo, et al.
Published: (2024)

EchoWorld: Learning Motion-Aware World Models for Echocardiography Probe Guidance
by: Yue, Yang, et al.
Published: (2025)

CheXWorld: Exploring Image World Modeling for Radiograph Representation Learning
by: Yue, Yang, et al.
Published: (2025)

UniTTA: Unified Benchmark and Versatile Framework Towards Realistic Test-Time Adaptation
by: Du, Chaoqun, et al.
Published: (2024)

DyDiT++: Diffusion Transformers with Timestep and Spatial Dynamics for Efficient Visual Generation
by: Zhao, Wangbo, et al.
Published: (2025)

SimA: Simple Softmax-free Attention for Vision Transformers
by: Koohpayegani, Soroush Abbasi, et al.
Published: (2022)

QuadricFormer: Scene as Superquadrics for 3D Semantic Occupancy Prediction
by: Zuo, Sicheng, et al.
Published: (2025)

Visual Adversarial Attacks and Defenses in the Physical World: A Survey
by: Wei, Xingxing, et al.
Published: (2022)

Rethinking Attention: Polynomial Alternatives to Softmax in Transformers
by: Saratchandran, Hemanth, et al.
Published: (2024)

Training an Open-Vocabulary Monocular 3D Object Detection Model without 3D Data
by: Huang, Rui, et al.
Published: (2024)

SIGMA: Bridging Structural and Distributional Gaps for Vision Foundation Model Adaptation
by: Xiong, Lingyu, et al.
Published: (2026)

Bridging the RGB-IR Gap: Consensus and Discrepancy Modeling for Text-Guided Multispectral Detection
by: Wu, Jiaqi, et al.
Published: (2026)

Probabilistic Contrastive Learning for Long-Tailed Visual Recognition
by: Du, Chaoqun, et al.
Published: (2024)

Meta-Semi: A Meta-learning Approach for Semi-supervised Learning
by: Wang, Yulin, et al.
Published: (2020)

Point Linguist Model: Segment Any Object via Bridged Large 3D-Language Model
by: Huang, Zhuoxu, et al.
Published: (2025)

ViT$^3$: Unlocking Test-Time Training in Vision
by: Han, Dongchen, et al.
Published: (2025)

Fast Personalized Text-to-Image Syntheses With Attention Injection
by: Zhang, Yuxuan, et al.
Published: (2024)