Decoupling Knowledge and Reasoning in Transformers: A Modular Architecture with Generalized Cross-Attention

Documenti analoghi

• DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion
  di: Chen, Yilong, et al.
  Pubblicazione: (2024)
• JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention
  di: Tian, Yuandong, et al.
  Pubblicazione: (2023)
• Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
  di: Ling Team, et al.
  Pubblicazione: (2025)
• Advancing General-Purpose Reasoning Models with Modular Gradient Surgery
  di: Cai, Min, et al.
  Pubblicazione: (2026)
• Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers
  di: Choi, Sehyun
  Pubblicazione: (2024)