MSign: An Optimizer Preventing Training Instability in Large Language Models via Stable Rank Restoration

Documenti analoghi

• Unifying back-propagation and forward-forward algorithms through model predictive control
  di: Ren, Lianhai, et al.
  Pubblicazione: (2024)
• Optimizing Large Language Model Training Using FP4 Quantization
  di: Wang, Ruizhe, et al.
  Pubblicazione: (2025)
• Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts
  di: Wang, Ruizhe, et al.
  Pubblicazione: (2025)
• From Generalization Analysis to Optimization Designs for State Space Models
  di: Liu, Fusheng, et al.
  Pubblicazione: (2024)
• StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization
  di: Wang, Shida, et al.
  Pubblicazione: (2023)