Extra-Merge: Tracing the Rank-1 Subspace of Model Merging in Language Model Pre-Training

Documenti analoghi

• The Stability of Singular Distribution: A Spectral Perspective on the Two-Phase Dynamics of Language Model Pre-training
  di: Zhang, Hongtao, et al.
  Pubblicazione: (2026)
• BSFA: Leveraging the Subspace Dichotomy to Accelerate Neural Network Training
  di: Zhou, Wenjie, et al.
  Pubblicazione: (2025)
• Model Merging in the Essential Subspace
  di: Li, Longhua, et al.
  Pubblicazione: (2026)
• Model Merging in Pre-training of Large Language Models
  di: Li, Yunshui, et al.
  Pubblicazione: (2025)
• When and Why Grouping Attention Heads Accelerates Muon Optimization
  di: Zhang, Hongtao, et al.
  Pubblicazione: (2026)