Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts

Documenti analoghi

• Cost-Optimal Grouped-Query Attention for Long-Context Modeling
  di: Chen, Yingfa, et al.
  Pubblicazione: (2025)
• StateX: Enhancing RNN Recall via Post-training State Expansion
  di: Shen, Xingyu, et al.
  Pubblicazione: (2025)
• MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
  di: MiniCPM Team, et al.
  Pubblicazione: (2026)
• $\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens
  di: Zhang, Xinrong, et al.
  Pubblicazione: (2024)
• Student-in-the-Loop Chain-of-Thought Distillation via Generation-Time Selection
  di: He, Chaoqun, et al.
  Pubblicazione: (2026)