How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective

Documenti analoghi

• Explicit Multi-head Attention for Inter-head Interaction in Large Language Models
  di: Peng, Runyu, et al.
  Pubblicazione: (2026)
• Data-free Weight Compress and Denoise for Large Language Models
  di: Peng, Runyu, et al.
  Pubblicazione: (2024)
• Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections
  di: Wang, Bo, et al.
  Pubblicazione: (2025)
• Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioning
  di: Zhang, Yechen, et al.
  Pubblicazione: (2026)
• When Attention Sink Emerges in Language Models: An Empirical View
  di: Gu, Xiangming, et al.
  Pubblicazione: (2024)