One LR Doesn't Fit All: Heavy-Tail Guided Layerwise Learning Rates for LLMs

Documenti analoghi

• AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
  di: He, Di, et al.
  Pubblicazione: (2025)
• Outlier-weighed Layerwise Sampling for LLM Fine-tuning
  di: Li, Pengxiang, et al.
  Pubblicazione: (2024)
• Infinite Width Models That Work: Why Feature Learning Doesn't Matter as Much as You Think
  di: Sernau, Luke
  Pubblicazione: (2024)
• Order Doesn't Matter, But Reasoning Does: Training LLMs with Order-Centric Augmentation
  di: He, Qianxi, et al.
  Pubblicazione: (2025)
• Bayesian Elicitation with LLMs: Model Size Helps, Extra "Reasoning" Doesn't Always
  di: Hobor, Luka, et al.
  Pubblicazione: (2026)