When Attention Sink Emerges in Language Models: An Empirical View

Documenti analoghi

• LightTransfer: Your Long-Context LLM is Secretly a Hybrid Model with Effortless Adaptation
  di: Zhang, Xuan, et al.
  Pubblicazione: (2024)
• Demystifying the Slash Pattern in Attention: The Role of RoPE
  di: Cheng, Yuan, et al.
  Pubblicazione: (2026)
• LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification
  di: Yang, Penghui, et al.
  Pubblicazione: (2025)
• Purifying Large Language Models by Ensembling a Small Language Model
  di: Li, Tianlin, et al.
  Pubblicazione: (2024)
• Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses
  di: Zheng, Xiaosen, et al.
  Pubblicazione: (2024)