Hyperparameter Transfer Enables Consistent Gains of Matrix-Preconditioned Optimizers Across Scales

Documenti analoghi

• Controllable Prompt Tuning For Balancing Group Distributional Robustness
  di: Phan, Hoang, et al.
  Pubblicazione: (2024)
• Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks
  di: Qiu, Shikai, et al.
  Pubblicazione: (2025)
• Large Language Models Are Zero-Shot Time Series Forecasters
  di: Gruver, Nate, et al.
  Pubblicazione: (2023)
• Transferring Knowledge from Large Foundation Models to Small Downstream Models
  di: Qiu, Shikai, et al.
  Pubblicazione: (2024)
• Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
  di: Marek, Martin, et al.
  Pubblicazione: (2026)