Transformers Don't Need LayerNorm at Inference Time: Scaling LayerNorm Removal to GPT-2 XL and the Implications for Mechanistic Interpretability

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Baroni, Luca, Khara, Galvin, Schaeffer, Joachim, Subkhankulov, Marat, Heimersheim, Stefan
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Internet

https://arxiv.org/abs/2507.02559