A Self-Attentive Meta-Optimizer with Group-Adaptive Learning Rates and Weight Decay

Documenti analoghi

• Theoretical Analysis of Sparse Optimization with Reparameterization, Weight Decay, and Adaptive Learning Rate
  di: Xu, Huangyu, et al.
  Pubblicazione: (2026)
• Deep Learning Optimization Using Self-Adaptive Weighted Auxiliary Variables
  di: Liu, Yaru, et al.
  Pubblicazione: (2025)
• Learning to Forget: Continual Learning with Adaptive Weight Decay
  di: Ramesh, Aditya A., et al.
  Pubblicazione: (2026)
• Weight Decay may matter more than muP for Learning Rate Transfer in Practice
  di: Kosson, Atli, et al.
  Pubblicazione: (2025)
• Optimal Learning-Rate Schedules under Functional Scaling Laws: Power Decay and Warmup-Stable-Decay
  di: Li, Binghui, et al.
  Pubblicazione: (2026)