Image de couverture de livre

When Does Removing LayerNorm Help? Activation Bounding as a Regime-Dependent Implicit Regularizer

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Verma, Lucky
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LayerNorm Induces Recency Bias in Transformer Decoders
par: Kim, Junu, et autres
Publié: (2025)

SLaNC: Static LayerNorm Calibration
par: Salmani, Mahsa, et autres
Publié: (2024)

Post-LayerNorm Is Back: Stable, ExpressivE, and Deep
par: Chen, Chen, et autres
Publié: (2026)

You can remove GPT2's LayerNorm by fine-tuning
par: Heimersheim, Stefan
Publié: (2024)

LayerNorm: A key component in parameter-efficient fine-tuning
par: ValizadehAslani, Taha, et autres
Publié: (2024)

Transformers Don't Need LayerNorm at Inference Time: Scaling LayerNorm Removal to GPT-2 XL and the Implications for Mechanistic Interpretability
par: Baroni, Luca, et autres
Publié: (2025)

Geometry and Dynamics of LayerNorm
par: Riechers, Paul M.
Publié: (2024)

On the Role of Attention Masks and LayerNorm in Transformers
par: Wu, Xinyi, et autres
Publié: (2024)

Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnostics
par: Verma, Lucky
Publié: (2026)

When Do "More Contexts" Help with Sarcasm Recognition?
par: Nimase, Ojas, et autres
Publié: (2024)

Concept Drift Guided LayerNorm Tuning for Efficient Multimodal Metaphor Identification
par: Qian, Wenhao, et autres
Publié: (2025)

MoETTA: Test-Time Adaptation Under Mixed Distribution Shifts with MoE-LayerNorm
par: Fan, Xiao, et autres
Publié: (2025)

GeoNorm: Unify Pre-Norm and Post-Norm with Geodesic Optimization
par: Zheng, Chuanyang, et autres
Publié: (2026)

SOLE: Hardware-Software Co-design of Softmax and LayerNorm for Efficient Transformer Inference
par: Wang, Wenxun, et autres
Publié: (2025)

Does Machine Unlearning Truly Remove Knowledge?
par: Chen, Haokun, et autres
Publié: (2025)

Implicit Word Reordering with Knowledge Distillation for Cross-Lingual Dependency Parsing
par: Li, Zhuoran, et autres
Publié: (2025)

Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
par: Nrusimha, Aniruddha, et autres
Publié: (2024)

How Transformers Learn Regular Language Recognition: A Theoretical Study on Training Dynamics and Implicit Bias
par: Huang, Ruiquan, et autres
Publié: (2025)

Does Representation Matter? Exploring Intermediate Layers in Large Language Models
par: Skean, Oscar, et autres
Publié: (2024)

Impact of Layer Norm on Memorization and Generalization in Transformers
par: Singhal, Rishi, et autres
Publié: (2025)

When and Why Does Unsupervised RL Succeed in Mathematical Reasoning? A Manifold Envelopment Perspective
par: Zhang, Zelin, et autres
Publié: (2026)

When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework
par: Xu, Zhen, et autres
Publié: (2025)

Does Visual Rendering Bypass Tokenization? Investigating Script-Tokenizer Misalignment in Pixel-Based Language Models
par: Susanto, Lucky, et autres
Publié: (2026)

When Does Multimodality Lead to Better Time Series Forecasting?
par: Zhang, Xiyuan, et autres
Publié: (2025)

When Does Metadata Conditioning (NOT) Work for Language Model Pre-Training? A Study with Context-Free Grammars
par: Higuchi, Rei, et autres
Publié: (2025)

SpecBound: Adaptive Bounded Self-Speculation with Layer-wise Confidence Calibration
par: Wen, Zhuofan, et autres
Publié: (2026)

Does Few-Shot Learning Help LLM Performance in Code Synthesis?
par: Xu, Derek, et autres
Publié: (2024)

When is Tree Search Useful for LLM Planning? It Depends on the Discriminator
par: Chen, Ziru, et autres
Publié: (2024)

When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models
par: Zheng, Mingqian, et autres
Publié: (2023)

On the Mathematical Relationship Between Layer Normalization and Dynamic Activation Functions
par: Stollenwerk, Felix
Publié: (2025)

When Does a Language Model Commit? A Finite-Answer Theory of Pre-Verbalization Commitment
par: Zhang, Long, et autres
Publié: (2026)

When Models Examine Themselves: Vocabulary-Activation Correspondence in Self-Referential Processing
par: Dadfar, Zachary Pedram
Publié: (2026)

HINT: Helping Ineffective Rollouts Navigate Towards Effectiveness
par: Wang, Xinyi, et autres
Publié: (2025)

When Does Demographic Information Help? Data and Modeling Regimes for Perspective-Aware Hate Speech Detection
par: Cai, Weibin, et autres
Publié: (2026)

PolyNorm: Few-Shot LLM-Based Text Normalization for Text-to-Speech
par: Wong, Michel, et autres
Publié: (2025)

Safety Training Persists Through Helpfulness Optimization in LLM Agents
par: Plaut, Benjamin
Publié: (2026)

Mimetic Initialization Helps State Space Models Learn to Recall
par: Trockman, Asher, et autres
Publié: (2024)

Skip-Connected Policy Optimization for Implicit Advantage
par: Teng, Fengwei, et autres
Publié: (2026)

Bootstrapping Language Models with DPO Implicit Rewards
par: Chen, Changyu, et autres
Publié: (2024)

Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents
par: Talokar, Nivya, et autres
Publié: (2026)