Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigation

Documenti analoghi

• Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
  di: Wu, Taiqiang, et al.
  Pubblicazione: (2024)
• MMFormalizer: Multimodal Autoformalization in the Wild
  di: Xiong, Jing, et al.
  Pubblicazione: (2026)
• XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression
  di: Su, Zunhai, et al.
  Pubblicazione: (2026)
• XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression
  di: Su, Zunhai, et al.
  Pubblicazione: (2026)
• KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs
  di: Su, Zunhai, et al.
  Pubblicazione: (2025)