Sparse Growing Transformer: Training-Time Sparse Depth Allocation via Progressive Attention Looping

Documenti analoghi

• DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion
  di: Chen, Yilong, et al.
  Pubblicazione: (2024)
• Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking
  di: Chen, Yilong, et al.
  Pubblicazione: (2025)
• Mixture of Universal Experts: Scaling Virtual Width via Depth-Width Transformation
  di: Chen, Yilong, et al.
  Pubblicazione: (2026)
• Mixture of Hidden-Dimensions Transformer
  di: Chen, Yilong, et al.
  Pubblicazione: (2024)
• NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time
  di: Chen, Yilong, et al.
  Pubblicazione: (2024)