Gradient Multi-Normalization for Stateless and Scalable LLM Training

Documenti analoghi

• SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training
  di: Ma, Chao, et al.
  Pubblicazione: (2024)
• Towards Efficient Optimizer Design for LLM via Structured Fisher Approximation with a Low-Rank Extension
  di: Gong, Wenbo, et al.
  Pubblicazione: (2025)
• MASP: Scalable GNN-based Planning for Multi-Agent Navigation
  di: Yang, Xinyi, et al.
  Pubblicazione: (2023)
• The Essential Role of Causality in Foundation World Models for Embodied AI
  di: Gupta, Tarun, et al.
  Pubblicazione: (2024)
• Does Your Optimizer Care How You Normalize? Normalization-Optimizer Coupling in LLM Training
  di: Abouzeid, Abdelrahman
  Pubblicazione: (2026)