SPLA: Block Sparse Plus Linear Attention for Long Context Modeling

Documenti analoghi

• RATTENTION: Towards the Minimal Sliding Window Size in Local-Global Attention Models
  di: Wang, Bailin, et al.
  Pubblicazione: (2025)
• RRAttention: Dynamic Block Sparse Attention via Per-Head Round-Robin Shifts for Long-Context Inference
  di: Liu, Siran, et al.
  Pubblicazione: (2026)
• MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
  di: MiniCPM Team, et al.
  Pubblicazione: (2026)
• Long-Context Generalization with Sparse Attention
  di: Vasylenko, Pavlo, et al.
  Pubblicazione: (2025)
• Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attention
  di: Xiao, Emily, et al.
  Pubblicazione: (2025)