Image de couverture de livre

Norm$\times$Direction: Restoring the Missing Query Norm in Vision Linear Attention

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Meng, Weikang, Luo, Yadan, Huo, Liangyu, Li, Yingjian, Wang, Yaowei, Li, Xin, Zhang, Zheng
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

MirrorLA: Reflecting Feature Map for Vision Linear Attention
par: Meng, Weikang, et autres
Publié: (2026)

STILL: Selecting Tokens for Intra-Layer Hybrid Attention to Linearize LLMs
par: Meng, Weikang, et autres
Publié: (2026)

ConjNorm: Tractable Density Estimation for Out-of-Distribution Detection
par: Peng, Bo, et autres
Publié: (2024)

PolaFormer: Polarity-aware Linear Attention for Vision Transformers
par: Meng, Weikang, et autres
Publié: (2025)

GeoNorm: Unify Pre-Norm and Post-Norm with Geodesic Optimization
par: Zheng, Chuanyang, et autres
Publié: (2026)

On the Role of Attention Masks and LayerNorm in Transformers
par: Wu, Xinyi, et autres
Publié: (2024)

Post-Norm can Resharpen Attention
par: Zsámboki, Pál, et autres
Publié: (2025)

GradientStabilizer:Fix the Norm, Not the Gradient
par: Huang, Tianjin, et autres
Publié: (2025)

Empirical Bound Information-Directed Sampling for Norm-Agnostic Bandits
par: Suder, Piotr M., et autres
Publié: (2025)

VL Norm: Rethink Loss Aggregation in RLVR
par: He, Zhiyuan, et autres
Publié: (2025)

Robust Capped lp-Norm Support Vector Ordinal Regression
par: Xiang, Haorui, et autres
Publié: (2024)

Muown: Row-Norm Control for Muon Optimization
par: Lion, Kai, et autres
Publié: (2026)

Norm-Bounded Low-Rank Adaptation
par: Wang, Ruigang, et autres
Publié: (2025)

Geometry and Dynamics of LayerNorm
par: Riechers, Paul M.
Publié: (2024)

Scalable Optimization in the Modular Norm
par: Large, Tim, et autres
Publié: (2024)

Muon Optimizes Under Spectral Norm Constraints
par: Chen, Lizhang, et autres
Publié: (2025)

SpanNorm: Reconciling Training Stability and Performance in Deep Transformers
par: Wang, Chao, et autres
Publié: (2026)

Scalable Frame-based Construction of Sociocultural NormBases for Socially-Aware Dialogues
par: Qu, Shilin, et autres
Publié: (2024)

The Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization
par: Kravatskiy, Alexey, et autres
Publié: (2025)

Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
par: Chezhegov, Savelii, et autres
Publié: (2024)

FlashNorm: Fast Normalization for Transformers
par: Graef, Nils, et autres
Publié: (2024)

Nuclear Norm Regularization for Deep Learning
par: Scarvelis, Christopher, et autres
Publié: (2024)

Optimal Scaling Needs Optimal Norm
par: Filatov, Oleg, et autres
Publié: (2025)

UnitNorm: Rethinking Normalization for Transformers in Time Series
par: Huang, Nan, et autres
Publié: (2024)

Greedy-Gnorm: A Gradient Matrix Norm-Based Alternative to Attention Entropy for Head Pruning
par: Guo, Yuxi, et autres
Publié: (2026)

Norm Augmented Graph AutoEncoders for Link Prediction
par: Liu, Yunhui, et autres
Publié: (2025)

Parametric $ρ$-Norm Scaling Calibration
par: Zhang, Siyuan, et autres
Publié: (2024)

On the Importance of Embedding Norms in Self-Supervised Learning
par: Draganov, Andrew, et autres
Publié: (2025)

Minimum-Norm Interpolation Under Covariate Shift
par: Mallinar, Neil, et autres
Publié: (2024)

Measuring Social Norms of Large Language Models
par: Yuan, Ye, et autres
Publié: (2024)

Discriminability-Driven Spatial-Channel Selection with Gradient Norm for Drone Signal OOD Detection
par: Feng, Chuhan, et autres
Publié: (2026)

Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization
par: Xie, Shuo, et autres
Publié: (2024)

Optimal Excess Risk Bounds for Empirical Risk Minimization on $p$-Norm Linear Regression
par: Hanchi, Ayoub El, et autres
Publié: (2023)

Old Optimizer, New Norm: An Anthology
par: Bernstein, Jeremy, et autres
Publié: (2024)

Distribution Estimation under the Infinity Norm
par: Kontorovich, Aryeh, et autres
Publié: (2024)

Neural Weight Norm = Kolmogorov Complexity
par: Musat, Tiberiu
Publié: (2026)

Transformers Don't Need LayerNorm at Inference Time: Scaling LayerNorm Removal to GPT-2 XL and the Implications for Mechanistic Interpretability
par: Baroni, Luca, et autres
Publié: (2025)

Batches Stabilize the Minimum Norm Risk in High Dimensional Overparameterized Linear Regression
par: Ioushua, Shahar Stein, et autres
Publié: (2023)

Optimal Sketching for Residual Error Estimation for Matrix and Vector Norms
par: Li, Yi, et autres
Publié: (2024)

Adaptive Norm-Based Regularization for Neural Networks
par: Qasim, Muhammad, et autres
Publié: (2026)