A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Qiu, Zihan, Huang, Zeyu, Wen, Kaiyue, Jin, Peng, Zheng, Bo, Zhou, Yuxin, Huang, Haofeng, Wang, Zekun, Li, Xiao, Zhang, Huaqing, Xu, Yang, Lian, Haoran, Zhang, Siqi, Men, Rui, Zhang, Jianwei, Titov, Ivan, Liu, Dayiheng, Zhou, Jingren, Lin, Junyang
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Internet

https://arxiv.org/abs/2601.22966