Copertina

Post-LayerNorm Is Back: Stable, ExpressivE, and Deep

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Chen, Chen, Wei, Lai
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

LayerNorm Induces Recency Bias in Transformer Decoders
di: Kim, Junu, et al.
Pubblicazione: (2025)

SLaNC: Static LayerNorm Calibration
di: Salmani, Mahsa, et al.
Pubblicazione: (2024)

You can remove GPT2's LayerNorm by fine-tuning
di: Heimersheim, Stefan
Pubblicazione: (2024)

LayerNorm: A key component in parameter-efficient fine-tuning
di: ValizadehAslani, Taha, et al.
Pubblicazione: (2024)

When Does Removing LayerNorm Help? Activation Bounding as a Regime-Dependent Implicit Regularizer
di: Verma, Lucky
Pubblicazione: (2026)

Geometry and Dynamics of LayerNorm
di: Riechers, Paul M.
Pubblicazione: (2024)

On the Role of Attention Masks and LayerNorm in Transformers
di: Wu, Xinyi, et al.
Pubblicazione: (2024)

Transformers Don't Need LayerNorm at Inference Time: Scaling LayerNorm Removal to GPT-2 XL and the Implications for Mechanistic Interpretability
di: Baroni, Luca, et al.
Pubblicazione: (2025)

MoETTA: Test-Time Adaptation Under Mixed Distribution Shifts with MoE-LayerNorm
di: Fan, Xiao, et al.
Pubblicazione: (2025)

GeoNorm: Unify Pre-Norm and Post-Norm with Geodesic Optimization
di: Zheng, Chuanyang, et al.
Pubblicazione: (2026)

Concept Drift Guided LayerNorm Tuning for Efficient Multimodal Metaphor Identification
di: Qian, Wenhao, et al.
Pubblicazione: (2025)

SOLE: Hardware-Software Co-design of Softmax and LayerNorm for Efficient Transformer Inference
di: Wang, Wenxun, et al.
Pubblicazione: (2025)

Kimi Linear: An Expressive, Efficient Attention Architecture
di: Kimi Team, et al.
Pubblicazione: (2025)

SpanNorm: Reconciling Training Stability and Performance in Deep Transformers
di: Wang, Chao, et al.
Pubblicazione: (2026)

Impact of Layer Norm on Memorization and Generalization in Transformers
di: Singhal, Rishi, et al.
Pubblicazione: (2025)

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization
di: Zhuo, Zhijian, et al.
Pubblicazione: (2025)

Verifier-Backed Hard Problem Generation for Mathematical Reasoning
di: Lai, Yuhang, et al.
Pubblicazione: (2026)

The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry
di: Zhang, Michael, et al.
Pubblicazione: (2024)

More Edits, More Stable: Understanding the Lifelong Normalization in Sequential Model Editing
di: Ma, Xin, et al.
Pubblicazione: (2026)

Provable Knowledge Acquisition and Extraction in One-Layer Transformers
di: Xu, Ruichen, et al.
Pubblicazione: (2025)

RAM-Net: Expressive Linear Attention with Selectively Addressable Memory
di: Xiao, Kaicheng, et al.
Pubblicazione: (2026)

Pause Tokens Strictly Increase the Expressivity of Constant-Depth Transformers
di: London, Charles, et al.
Pubblicazione: (2025)

Interactive and Expressive Code-Augmented Planning with Large Language Models
di: Liu, Anthony Z., et al.
Pubblicazione: (2024)

Taipan: Efficient and Expressive State Space Language Models with Selective Attention
di: Van Nguyen, Chien, et al.
Pubblicazione: (2024)

Learning to (Learn at Test Time): RNNs with Expressive Hidden States
di: Sun, Yu, et al.
Pubblicazione: (2024)

Stochastic Attention: Connectome-Inspired Randomized Routing for Expressive Linear-Time Attention
di: Jin, Zehao, et al.
Pubblicazione: (2026)

An Algebraic View of the Expressivity of Recurrent Language Models
di: Nowak, Franz, et al.
Pubblicazione: (2026)

Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge
di: Chen, Yan-Lun, et al.
Pubblicazione: (2025)

WILDCHAT-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training
di: Feuer, Benjamin, et al.
Pubblicazione: (2025)

More Expressive Attention with Negative Weights
di: Lv, Ang, et al.
Pubblicazione: (2024)

The Expressive Power of Low-Rank Adaptation
di: Zeng, Yuchen, et al.
Pubblicazione: (2023)

Decoding Linguistic Nuances in Mental Health Text Classification Using Expressive Narrative Stories
di: Tang, Jinwen, et al.
Pubblicazione: (2024)

MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression
di: Sun, Libo, et al.
Pubblicazione: (2026)

Olmo Hybrid: From Theory to Practice and Back
di: Merrill, William, et al.
Pubblicazione: (2026)

Can Post-Training Transform LLMs into Causal Reasoners?
di: Chen, Junqi, et al.
Pubblicazione: (2026)

The Expressive Power of Low Precision Softmax Transformers with (Summarized) Chain-of-Thought
di: Brösamle, Moritz, et al.
Pubblicazione: (2026)

PolyNorm: Few-Shot LLM-Based Text Normalization for Text-to-Speech
di: Wong, Michel, et al.
Pubblicazione: (2025)

Efficient Post-Training Pruning of Large Language Models with Statistical Correction
di: Yu, Peiqi, et al.
Pubblicazione: (2026)

BEATS: Optimizing LLM Mathematical Capabilities with BackVerify and Adaptive Disambiguate based Efficient Tree Search
di: Sun, Linzhuang, et al.
Pubblicazione: (2024)

FBQuant: FeedBack Quantization for Large Language Models
di: Liu, Yijiang, et al.
Pubblicazione: (2025)