Image de couverture de livre

On the Nonlinearity of Layer Normalization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Ni, Yunhao, Guo, Yuxin, Jia, Junlong, Huang, Lei
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Parallel Layer Normalization for Universal Approximation
par: Ni, Yunhao, et autres
Publié: (2025)

Enjoy Your Layer Normalization with the Computational Efficiency of RMSNorm
par: Guo, Yuxin, et autres
Publié: (2026)

TimeGMM: Single-Pass Probabilistic Forecasting via Adaptive Gaussian Mixture Models with Reversible Normalization
par: Liu, Lei, et autres
Publié: (2026)

Personalized Observation Normalization for Federated Reinforcement Learning in Simulation Environments with Heterogeneity
par: Pang, Yiran, et autres
Publié: (2026)

The Hidden Power of Normalization Layers in Neural Networks: Exponential Capacity Control
par: Than, Khoat
Publié: (2025)

Stability of Transformers under Layer Normalization
par: Kan, Kelvin, et autres
Publié: (2025)

Enabling Causal Discovery in Post-Nonlinear Models with Normalizing Flows
par: Hoang, Nu, et autres
Publié: (2024)

On the Role of Transformer Feed-Forward Layers in Nonlinear In-Context Learning
par: Sun, Haoyuan, et autres
Publié: (2025)

IBNorm: Information-Bottleneck Inspired Normalization for Representation Learning
par: Zou, Xiandong, et autres
Publié: (2025)

Peri-LN: Revisiting Normalization Layer in the Transformer Architecture
par: Kim, Jeonghoon, et autres
Publié: (2025)

Context Normalization Layer with Applications
par: Faye, Bilal, et autres
Publié: (2023)

Enhancing Convergence in Federated Learning: A Contribution-Aware Asynchronous Approach
par: Xu, Changxin, et autres
Publié: (2024)

Geometric Interpretation of Layer Normalization and a Comparative Analysis with RMSNorm
par: Gupta, Akshat, et autres
Publié: (2024)

On the Mathematical Relationship Between Layer Normalization and Dynamic Activation Functions
par: Stollenwerk, Felix
Publié: (2025)

Non-Neighbors Also Matter to Kriging: A New Contrastive-Prototypical Learning
par: Li, Zhishuai, et autres
Publié: (2024)

PolicyLong: Towards On-Policy Context Extension
par: Jia, Junlong, et autres
Publié: (2026)

A Unified Framework for Generative Data Augmentation: A Comprehensive Survey
par: Chen, Yunhao, et autres
Publié: (2023)

Conda: Column-Normalized Adam for Training Large Language Models Faster
par: Wang, Junjie, et autres
Publié: (2025)

Disentangled Graph Autoencoder for Treatment Effect Estimation
par: Fan, Di, et autres
Publié: (2024)

Testability of Instrumental Variables in Additive Nonlinear, Non-Constant Effects Models
par: Guo, Xichen, et autres
Publié: (2024)

Learning Hierarchical Polynomials of Multiple Nonlinear Features with Three-Layer Networks
par: Fu, Hengyu, et autres
Publié: (2024)

PSPO*: An Effective Process-supervised Policy Optimization for Reasoning Alignment
par: Li, Jiawei, et autres
Publié: (2024)

Layer-Aware Influence for Online Data Valuation Estimation
par: Yang, Ziao, et autres
Publié: (2025)

Non-Normal Diffusion Models
par: Li, Henry
Publié: (2024)

Does Your Optimizer Care How You Normalize? Normalization-Optimizer Coupling in LLM Training
par: Abouzeid, Abdelrahman
Publié: (2026)

E-CGL: An Efficient Continual Graph Learner
par: Guo, Jianhao, et autres
Publié: (2024)

Knowledge Graph Embedding by Normalizing Flows
par: Xiao, Changyi, et autres
Publié: (2024)

Scaling CrossQ with Weight Normalization
par: Palenicek, Daniel, et autres
Publié: (2025)

Normalized Architectures are Natively 4-Bit
par: Fishman, Maxim, et autres
Publié: (2026)

Learning Rate Transfer in Normalized Transformers
par: Shigida, Boris, et autres
Publié: (2026)

On the Weight Dynamics of Deep Normalized Networks
par: Mehmeti-Göpel, Christian H. X. Ali, et autres
Publié: (2023)

Amortized Sampling with Transferable Normalizing Flows
par: Tan, Charlie B., et autres
Publié: (2025)

BNPO: Beta Normalization Policy Optimization
par: Xiao, Changyi, et autres
Publié: (2025)

ANAct: Adaptive Normalization for Activation Functions
par: Peiwen, Yuan, et autres
Publié: (2022)

When Normality Shifts: Risk-Aware Test-Time Adaptation for Unsupervised Tabular Anomaly Detection
par: Huang, Wei, et autres
Publié: (2026)

Integration of Mamba and Transformer -- MAT for Long-Short Range Time Series Forecasting with Application to Weather Dynamics
par: Zhang, Wenqing, et autres
Publié: (2024)

GRANOLA: Adaptive Normalization for Graph Neural Networks
par: Eliasof, Moshe, et autres
Publié: (2024)

Self-Normalized Resets for Plasticity in Continual Learning
par: Farias, Vivek F., et autres
Publié: (2024)

Continuous-Time Analysis of Adaptive Optimization and Normalization
par: Gould, Rhys, et autres
Publié: (2024)

Normalization and effective learning rates in reinforcement learning
par: Lyle, Clare, et autres
Publié: (2024)