HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization

Documenti analoghi

• Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models
  di: Zhuo, Zhijian, et al.
  Pubblicazione: (2024)
• Scale-Distribution Decoupling: Enabling Stable and Effective Training of Large Language Models
  di: Wang, Ya, et al.
  Pubblicazione: (2025)
• Efficient Pretraining Length Scaling
  di: Wu, Bohong, et al.
  Pubblicazione: (2025)
• SpanNorm: Reconciling Training Stability and Performance in Deep Transformers
  di: Wang, Chao, et al.
  Pubblicazione: (2026)
• Understanding In-Context Learning Beyond Transformers: An Investigation of State Space and Hybrid Architectures
  di: Wang, Shenran, et al.
  Pubblicazione: (2025)