Don't be lazy: CompleteP enables compute-efficient deep transformers

Documenti analoghi

• Sparse maximal update parameterization: A holistic approach to sparse training dynamics
  di: Dey, Nolan, et al.
  Pubblicazione: (2024)
• Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs
  di: Bergsma, Shane, et al.
  Pubblicazione: (2025)
• Hyperparameter Transfer with Mixture-of-Expert Layers
  di: Jiang, Tianze, et al.
  Pubblicazione: (2026)
• Disordered Dynamics in High Dimensions: Connections to Random Matrices and Machine Learning
  di: Bordelon, Blake, et al.
  Pubblicazione: (2026)
• Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer
  di: Bordelon, Blake, et al.
  Pubblicazione: (2025)