Spectral-factorized Positive-definite Curvature Learning for NN Training

Documenti analoghi

• Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
  di: Lin, Wu, et al.
  Pubblicazione: (2024)
• Kronecker-factored Approximate Curvature (KFAC) From Scratch
  di: Dangel, Felix, et al.
  Pubblicazione: (2025)
• Position: Curvature Matrices Should Be Democratized via Linear Operators
  di: Dangel, Felix, et al.
  Pubblicazione: (2025)
• Understanding and Improving Shampoo and SOAP via Kullback-Leibler Minimization
  di: Lin, Wu, et al.
  Pubblicazione: (2025)
• Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC
  di: Lin, Wu, et al.
  Pubblicazione: (2023)