Image de couverture de livre

Local Linear Attention: An Optimal Interpolation of Linear and Softmax Attention For Test-Time Regression

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zuo, Yifei, Yin, Yutong, Zeng, Zhichen, Li, Ang, Zhu, Banghua, Wang, Zhaoran
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Parallax: Parameterized Local Linear Attention for Language Modeling
par: Zuo, Yifei, et autres
Publié: (2026)

MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map
par: Chou, Yuhong, et autres
Publié: (2024)

Softmax Linear Attention: Reclaiming Global Competition
par: Xu, Mingwei, et autres
Publié: (2026)

Superiority of Multi-Head Attention in In-Context Linear Regression
par: Cui, Yingqian, et autres
Publié: (2024)

Universal Approximation with Softmax Attention
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2025)

Are Transformers Able to Reason by Connecting Separated Knowledge in Training Data?
par: Yin, Yutong, et autres
Publié: (2025)

Softmax-free Linear Transformers
par: Lu, Jiachen, et autres
Publié: (2022)

Exact Linear Attention
par: Ou, Weinuo
Publié: (2026)

Kaczmarz Linear Attention
par: Zou, Jiaxuan, et autres
Publié: (2026)

Efficient Linear Attention for Multivariate Time Series Modeling via Entropy Equality
par: Zhang, Mingtao, et autres
Publié: (2025)

Test-Time Training with KV Binding Is Secretly Linear Attention
par: Liu, Junchen, et autres
Publié: (2026)

Transolver is a Linear Transformer: Revisiting Physics-Attention through the Lens of Linear Attention
par: Hu, Wenjie, et autres
Publié: (2025)

Scalable-Softmax Is Superior for Attention
par: Nakanishi, Ken M.
Publié: (2025)

RACE Attention: A Strictly Linear-Time Attention Layer for Training on Outrageously Large Contexts
par: Joshi, Sahil, et autres
Publié: (2025)

In-Context Learning in Linear vs. Quadratic Attention Models: An Empirical Study on Regression Tasks
par: Goel, Ayush, et autres
Publié: (2026)

Adaptive Memory Decay for Log-Linear Attention
par: Amin, Yaxita, et autres
Publié: (2026)

Higher-order Linear Attention
par: Zhang, Yifan, et autres
Publié: (2025)

On the Invariants of Softmax Attention
par: Lee, Wonsuk
Publié: (2026)

Enhancing Linear Attention with Residual Learning
par: Lai, Xunhao, et autres
Publié: (2025)

Minimalist Softmax Attention Provably Learns Constrained Boolean Functions
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2025)

FAME: Adaptive Functional Attention with Expert Routing for Function-on-Function Regression
par: Gao, Yifei, et autres
Publié: (2025)

KVBuffer: IO-aware Serving for Linear Attention
par: Zou, Longwei, et autres
Publié: (2026)

Linear Attention is Enough in Spatial-Temporal Forecasting
par: Ning, Xinyu
Publié: (2024)

State Rank Dynamics in Linear Attention LLMs
par: Sun, Ao, et autres
Publié: (2026)

Linear Attention for Efficient Bidirectional Sequence Modeling
par: Afzal, Arshia, et autres
Publié: (2025)

Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs
par: Zhu, Kan, et autres
Publié: (2025)

Learning Linear Attention in Polynomial Time
par: Yau, Morris, et autres
Publié: (2024)

Degrees of Freedom for Linear Attention: Distilling Softmax Attention with Optimal Feature Efficiency
par: Nishikawa, Naoki, et autres
Publié: (2025)

Implicit Regularization of Gradient Flow on One-Layer Softmax Attention
par: Sheen, Heejune, et autres
Publié: (2024)

Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving
par: Zhou, Qihui, et autres
Publié: (2025)

Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression
par: Chen, Xingwu, et autres
Publié: (2025)

Tiled Flash Linear Attention: More Efficient Linear RNN and xLSTM Kernels
par: Beck, Maximilian, et autres
Publié: (2025)

Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality
par: Chen, Siyu, et autres
Publié: (2024)

Why Softmax Attention Outperforms Linear Attention
par: Deng, Yichuan, et autres
Publié: (2023)

FLASH-D: FlashAttention with Hidden Softmax Division
par: Alexandridis, Kosmas, et autres
Publié: (2025)

Forgetting Transformer: Softmax Attention with a Forget Gate
par: Lin, Zhixuan, et autres
Publié: (2025)

Beyond Linearity in Attention Projections: The Case for Nonlinear Queries
par: Karbevski, Marko
Publié: (2026)

ZeroS: Zero-Sum Linear Attention for Efficient Transformers
par: Lu, Jiecheng, et autres
Publié: (2026)

Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspective
par: Yan, Fanqi, et autres
Publié: (2025)

Scaling Attention to Very Long Sequences in Linear Time with Wavelet-Enhanced Random Spectral Attention (WERSA)
par: Dentamaro, Vincenzo
Publié: (2025)