Optimal Condition for Initialization Variance in Deep Neural Networks: An SGD Dynamics Perspective

Documenti analoghi

• Weight Initialization and Variance Dynamics in Deep Neural Networks and Large Language Models
  di: Han, Yankun
  Pubblicazione: (2025)
• SGD with Partial Hessian for Deep Neural Networks Optimization
  di: Sun, Ying, et al.
  Pubblicazione: (2024)
• From SGD to Spectra: A Theory of Neural Network Weight Dynamics
  di: Olsen, Brian Richard, et al.
  Pubblicazione: (2025)
• Exploring and Improving Initialization for Deep Graph Neural Networks: A Signal Propagation Perspective
  di: Wang, Senmiao, et al.
  Pubblicazione: (2025)
• SGD for Variational Inference: Tackling Unbounded Variance via Preconditioning and Dynamic Batching
  di: Labarrière, Hippolyte, et al.
  Pubblicazione: (2026)