Isotropic Curvature Model for Understanding Deep Learning Optimization: Is Gradient Orthogonalization Optimal?

Documenti analoghi

• The Newton-Muon Optimizer
  di: Du, Zhehang, et al.
  Pubblicazione: (2026)
• Uncovering Symmetry Transfer in Large Language Models via Layer-Peeled Optimization
  di: Du, Zhehang, et al.
  Pubblicazione: (2026)
• Understanding Optimization in Deep Learning with Central Flows
  di: Cohen, Jeremy M., et al.
  Pubblicazione: (2024)
• Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers
  di: Lau, Tim Tsz-Kit, et al.
  Pubblicazione: (2026)
• On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization
  di: Ma, Shaocong, et al.
  Pubblicazione: (2025)