Cover Image

Softmax-free Linear Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lu, Jiachen, Zhang, Junge, Zhu, Xiatian, Feng, Jianfeng, Xiang, Tao, Zhang, Li
Format:	Preprint
Published:	2022
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems
by: Hoffmann, David T., et al.
Published: (2023)

Stronger Normalization-Free Transformers
by: Chen, Mingzhi, et al.
Published: (2025)

Transformers without Normalization
by: Zhu, Jiachen, et al.
Published: (2025)

SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention
by: Zhang, Jintao, et al.
Published: (2025)

Improving Long-Tailed Object Detection with Balanced Group Softmax and Metric Learning
by: Gaba, Satyam
Published: (2025)

Training-free Linear Image Inverses via Flows
by: Pokle, Ashwini, et al.
Published: (2023)

VDSB-GWSyn: Diffusion Schrödinger Bridge for Controllable and Anatomically Feasible Guidewire Synthesis in Coronary Angiography
by: Tang, Haoyuan, et al.
Published: (2026)

Shiva-DiT: Residual-Based Differentiable Top-$k$ Selection for Efficient Diffusion Transformers
by: Zhang, Jiaji, et al.
Published: (2026)

Low-Resource Crop Classification from Multi-Spectral Time Series Using Lossless Compressors
by: Cheng, Wei, et al.
Published: (2024)

SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference
by: Zhang, Jintao, et al.
Published: (2025)

AsyCo: An Asymmetric Dual-task Co-training Model for Partial-label Learning
by: Li, Beibei, et al.
Published: (2024)

Tracing Representation Progression: Analyzing and Enhancing Layer-Wise Similarity
by: Jiang, Jiachen, et al.
Published: (2024)

Scaling Diffusion Transformers Efficiently via $μ$P
by: Zheng, Chenyu, et al.
Published: (2025)

Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learning
by: Hu, Zixuan, et al.
Published: (2023)

Geometrically Constrained Stenosis Editing in Coronary Angiography via Entropic Optimal Transport
by: Li, Jialin, et al.
Published: (2026)

Precipitation Nowcasting Using Diffusion Transformer with Causal Attention
by: Li, ChaoRong, et al.
Published: (2024)

i-MAE: Are Latent Representations in Masked Autoencoders Linearly Separable?
by: Zhang, Kevin, et al.
Published: (2022)

SLA2: Sparse-Linear Attention with Learnable Routing and QAT
by: Zhang, Jintao, et al.
Published: (2026)

Rethinking Attention: Polynomial Alternatives to Softmax in Transformers
by: Saratchandran, Hemanth, et al.
Published: (2024)

Variance-Covariance Regularization Improves Representation Learning
by: Zhu, Jiachen, et al.
Published: (2023)

FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximation
by: Liu, Dong, et al.
Published: (2025)

Fairness-aware Vision Transformer via Debiased Self-Attention
by: Qiang, Yao, et al.
Published: (2023)

Towards Precise Scaling Laws for Video Diffusion Transformers
by: Yin, Yuanyang, et al.
Published: (2024)

Astra: General Interactive World Model with Autoregressive Denoising
by: Zhu, Yixuan, et al.
Published: (2025)

Sparse Model Inversion: Efficient Inversion of Vision Transformers for Data-Free Applications
by: Hu, Zixuan, et al.
Published: (2025)

CGFformer: Cluster-Guidance Frequency Transformer for Pansharpening
by: Zhou, Zijian, et al.
Published: (2026)

HART: Efficient Visual Generation with Hybrid Autoregressive Transformer
by: Tang, Haotian, et al.
Published: (2024)

Practical Continual Forgetting for Pre-trained Vision Models
by: Zhao, Hongbo, et al.
Published: (2025)

Adaptive Prediction Ensemble: Improving Out-of-Distribution Generalization of Motion Forecasting
by: Li, Jinning, et al.
Published: (2024)

MatchDiffusion: Training-free Generation of Match-cuts
by: Pardo, Alejandro, et al.
Published: (2024)

Transformer-Based Classification Outcome Prediction for Multimodal Stroke Treatment
by: Ma, Danqing, et al.
Published: (2024)

Vision Transformers: From Semantic Segmentation to Dense Prediction
by: Zhang, Li, et al.
Published: (2022)

SpeCa: Accelerating Diffusion Transformers with Speculative Feature Caching
by: Liu, Jiacheng, et al.
Published: (2025)

H$_{2}$OT: Hierarchical Hourglass Tokenizer for Efficient Video Pose Transformers
by: Li, Wenhao, et al.
Published: (2025)

DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learning
by: Bai, Sikai, et al.
Published: (2024)

Robust Representation Consistency Model via Contrastive Denoising
by: Lei, Jiachen, et al.
Published: (2025)

BranchGRPO: Stable and Efficient GRPO with Structured Branching in Diffusion Models
by: Li, Yuming, et al.
Published: (2025)

TAP-ViTs: Task-Adaptive Pruning for On-Device Deployment of Vision Transformers
by: Wang, Zhibo, et al.
Published: (2026)

Analyzing Fine-Grained Alignment and Enhancing Vision Understanding in Multimodal Language Models
by: Jiang, Jiachen, et al.
Published: (2025)

TTF-VLA: Temporal Token Fusion via Pixel-Attention Integration for Vision-Language-Action Models
by: Liu, Chenghao, et al.
Published: (2025)