Cover Image

Agent Attention: On the Integration of Softmax and Linear Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Han, Dongchen, Ye, Tianzhu, Han, Yizeng, Xia, Zhuofan, Pan, Siyuan, Wan, Pengfei, Song, Shiji, Huang, Gao
Format:	Preprint
Published:	2023
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Bridging the Divide: Reconsidering Softmax and Linear Attention
by: Han, Dongchen, et al.
Published: (2024)

Demystify Mamba in Vision: A Linear Attention Perspective
by: Han, Dongchen, et al.
Published: (2024)

GSVA: Generalized Segmentation via Multimodal Large Language Models
by: Xia, Zhuofan, et al.
Published: (2023)

Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediators
by: Pu, Yifan, et al.
Published: (2024)

Linear-Time Global Visual Modeling without Explicit Attention
by: He, Ruize, et al.
Published: (2026)

Step by Step Network
by: Han, Dongchen, et al.
Published: (2025)

Vision Transformers are Circulant Attention Learners
by: Han, Dongchen, et al.
Published: (2025)

Latency-aware Unified Dynamic Networks for Efficient Image Recognition
by: Han, Yizeng, et al.
Published: (2023)

DyFADet: Dynamic Feature Aggregation for Temporal Action Detection
by: Yang, Le, et al.
Published: (2024)

GRA: Detecting Oriented Objects through Group-wise Rotating and Attention
by: Wang, Jiangshan, et al.
Published: (2024)

SoLA-Vision: Fine-grained Layer-wise Linear Softmax Hybrid Attention
by: Li, Ruibang, et al.
Published: (2026)

EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training
by: Wang, Yulin, et al.
Published: (2024)

SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning
by: Du, Chaoqun, et al.
Published: (2024)

SALAD: Achieve High-Sparsity Attention via Efficient Linear Attention Tuning for Video Diffusion Transformer
by: Fang, Tongcheng, et al.
Published: (2026)

Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials
by: Pu, Yifan, et al.
Published: (2025)

EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction
by: Cai, Han, et al.
Published: (2022)

SimA: Simple Softmax-free Attention for Vision Transformers
by: Koohpayegani, Soroush Abbasi, et al.
Published: (2022)

Adapting Vision-Language Model with Fine-grained Semantics for Open-Vocabulary Segmentation
by: Chng, Yong Xien, et al.
Published: (2024)

Rethinking Attention: Polynomial Alternatives to Softmax in Transformers
by: Saratchandran, Hemanth, et al.
Published: (2024)

Fast Personalized Text-to-Image Syntheses With Attention Injection
by: Zhang, Yuxuan, et al.
Published: (2024)

Mask Grounding for Referring Image Segmentation
by: Chng, Yong Xien, et al.
Published: (2023)

LINA: Linear Autoregressive Image Generative Models with Continuous Tokens
by: Wang, Jiahao, et al.
Published: (2026)

Attention to Detail: Global-Local Attention for High-Resolution AI-Generated Image Detection
by: Han, Lawrence
Published: (2026)

A-SDM: Accelerating Stable Diffusion through Redundancy Removal and Performance Optimization
by: Zhu, Jinchao, et al.
Published: (2023)

A-SDM: Accelerating Stable Diffusion through Model Assembly and Feature Inheritance Strategies
by: Zhu, Jinchao, et al.
Published: (2024)

Rectifying Magnitude Neglect in Linear Attention
by: Fan, Qihang, et al.
Published: (2025)

XAttention: Block Sparse Attention with Antidiagonal Scoring
by: Xu, Ruyi, et al.
Published: (2025)

VMoBA: Mixture-of-Block Attention for Video Diffusion Models
by: Wu, Jianzong, et al.
Published: (2025)

Exploring contextual modeling with linear complexity for point cloud segmentation
by: Chng, Yong Xien, et al.
Published: (2024)

Breaking the Low-Rank Dilemma of Linear Attention
by: Fan, Qihang, et al.
Published: (2024)

Dynamic Diffusion Transformer
by: Zhao, Wangbo, et al.
Published: (2024)

Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation
by: Zhao, Wangbo, et al.
Published: (2024)

Linear Attention Modeling for Learned Image Compression
by: Feng, Donghui, et al.
Published: (2025)

MCA: Moment Channel Attention Networks
by: Jiang, Yangbo, et al.
Published: (2024)

Few-Step Distillation for Text-to-Image Generation: A Practical Guide
by: Pu, Yifan, et al.
Published: (2025)

EchoWorld: Learning Motion-Aware World Models for Echocardiography Probe Guidance
by: Yue, Yang, et al.
Published: (2025)

CheXWorld: Exploring Image World Modeling for Radiograph Representation Learning
by: Yue, Yang, et al.
Published: (2025)

MODA: MOdular Duplex Attention for Multimodal Perception, Cognition, and Emotion Understanding
by: Zhang, Zhicheng, et al.
Published: (2025)

Attention Hijackers: Detect and Disentangle Attention Hijacking in LVLMs for Hallucination Mitigation
by: Chen, Beitao, et al.
Published: (2025)

FullDiT: Multi-Task Video Generative Foundation Model with Full Attention
by: Ju, Xuan, et al.
Published: (2025)