SpargeAttention2: Trainable Sparse Attention via Hybrid Top-k+Top-p Masking and Distillation Fine-Tuning

Documenti analoghi

• SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference
  di: Zhang, Jintao, et al.
  Pubblicazione: (2025)
• SageBwd: A Trainable Low-bit Attention
  di: Zhang, Jintao, et al.
  Pubblicazione: (2026)
• Trainable Dynamic Mask Sparse Attention
  di: Shi, Jingze, et al.
  Pubblicazione: (2025)
• HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference
  di: Gong, Ping, et al.
  Pubblicazione: (2025)
• SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention
  di: Zhang, Jintao, et al.
  Pubblicazione: (2025)