Image de couverture de livre

Enjoy Your Layer Normalization with the Computational Efficiency of RMSNorm

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Guo, Yuxin, Yue, Yihao, Ni, Yunhao, Ruan, Yizhou, Luo, Jie, Wu, Wenjun, Huang, Lei
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Parallel Layer Normalization for Universal Approximation
par: Ni, Yunhao, et autres
Publié: (2025)

On the Nonlinearity of Layer Normalization
par: Ni, Yunhao, et autres
Publié: (2024)

Geometric Interpretation of Layer Normalization and a Comparative Analysis with RMSNorm
par: Gupta, Akshat, et autres
Publié: (2024)

Free Privacy Protection for Wireless Federated Learning: Enjoy It or Suffer from It?
par: Li, Weicai, et autres
Publié: (2025)

Modulate Your Spectrum in Self-Supervised Learning
par: Weng, Xi, et autres
Publié: (2023)

An Investigation of Batch Normalization in Off-Policy Actor-Critic Algorithms
par: Wang, Li, et autres
Publié: (2025)

A Geometric Analysis of Sign-Magnitude Asymmetry in a ReLU + RMSNorm Block under Ternary Quantization
par: Dong, Lei
Publié: (2026)

An Extra RMSNorm is All You Need for Fine Tuning to 1.58 Bits
par: Steinmetz, Cody, et autres
Publié: (2025)

Design of Restricted Normalizing Flow towards Arbitrary Stochastic Policy with Computational Efficiency
par: Kobayashi, Taisuke, et autres
Publié: (2024)

DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representations
par: Zhu, Guogang, et autres
Publié: (2024)

Looped Transformers with Layer Normalization Provably Learn the Power Method
par: Wu, Lyumin, et autres
Publié: (2026)

ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy
par: Yang, Zonghan, et autres
Publié: (2024)

Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombination
par: Hu, Ming, et autres
Publié: (2023)

Smoothing-Based Conformal Prediction for Balancing Efficiency and Interpretability
par: Zheng, Mingyi, et autres
Publié: (2025)

Enjoying Non-linearity in Multinomial Logistic Bandits: A Minimax-Optimal Algorithm
par: Boudart, Pierre, et autres
Publié: (2025)

Spooky Action at a Distance: Normalization Layers Enable Side-Channel Spatial Communication
par: Pfrommer, Samuel, et autres
Publié: (2025)

RLLaVA: An RL-central Framework for Language and Vision Assistants
par: Zhao, Lei, et autres
Publié: (2025)

Does Your Optimizer Care How You Normalize? Normalization-Optimizer Coupling in LLM Training
par: Abouzeid, Abdelrahman
Publié: (2026)

Cauchy-Schwarz Fairness Regularizer
par: Liu, Yezi, et autres
Publié: (2025)

Hardware-Efficient Softmax and Layer Normalization with Guaranteed Normalization for Edge Devices
par: Choi, Dawon, et autres
Publié: (2026)

Surrogate Ensemble in Expensive Multi-Objective Optimization via Deep Q-Learning
par: Wu, Yuxin, et autres
Publié: (2026)

The Final Layer Holds the Key: A Unified and Efficient GNN Calibration Framework
par: Huang, Jincheng, et autres
Publié: (2025)

Determining Layer-wise Sparsity for Large Language Models Through a Theoretical Perspective
par: Huang, Weizhong, et autres
Publié: (2025)

Understanding the Role of Layer Normalization in Label-Skewed Federated Learning
par: Zhang, Guojun, et autres
Publié: (2023)

SAN: Inducing Metrizability of GAN with Discriminative Normalized Linear Layer
par: Takida, Yuhta, et autres
Publié: (2023)

Recent Advances on Machine Learning for Computational Fluid Dynamics: A Survey
par: Wang, Haixin, et autres
Publié: (2024)

Understanding Large Language Models in Your Pockets: Performance Study on COTS Mobile Devices
par: Xiao, Jie, et autres
Publié: (2024)

Efficient Link Prediction via GNN Layers Induced by Negative Sampling
par: Wang, Yuxin, et autres
Publié: (2023)

Accelerating Time Series Foundation Models with Speculative Decoding
par: Subbaraman, Pranav, et autres
Publié: (2025)

Exploiting Layer Normalization Fine-tuning in Visual Transformer Foundation Models for Classification
par: Tan, Zhaorui, et autres
Publié: (2025)

Enabling Group Fairness in Graph Unlearning via Bi-level Debiasing
par: Liu, Yezi, et autres
Publié: (2025)

Your Graph Recommender is Provably a Single-view Graph Contrastive Learning
par: Yang, Wenjie, et autres
Publié: (2024)

Renormalizable Spectral-Shell Dynamics as the Origin of Neural Scaling Laws
par: Zhang, Yizhou
Publié: (2025)

A Generalized Spectral Framework to Expain Neural Scaling and Compression Dynamics
par: Zhang, Yizhou
Publié: (2025)

Stability of Transformers under Layer Normalization
par: Kan, Kelvin, et autres
Publié: (2025)

Recover-to-Forget: Gradient Reconstruction from LoRA for Efficient LLM Unlearning
par: Liu, Yezi, et autres
Publié: (2025)

LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples
par: Liu, Yezi, et autres
Publié: (2025)

Learning to Normalize on the SPD Manifold under Bures-Wasserstein Geometry
par: Wang, Rui, et autres
Publié: (2025)

Context Normalization Layer with Applications
par: Faye, Bilal, et autres
Publié: (2023)

DriftGuard: Mitigating Asynchronous Data Drift in Federated Learning
par: Han, Yizhou, et autres
Publié: (2026)