Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers

Documenti analoghi

• Annotation Entropy Predicts Per-Example Learning Dynamics in LoRA Fine-Tuning
  di: Steele, Brady
  Pubblicazione: (2026)
• Identifying Policy Gradient Subspaces
  di: Schneider, Jan, et al.
  Pubblicazione: (2024)
• Low-Rank Adapters Initialization via Gradient Surgery for Continual Learning
  di: Pasquali, Joana, et al.
  Pubblicazione: (2026)
• Fractional Policy Gradients: Reinforcement Learning with Long-Term Memory
  di: Pawar, Urvi, et al.
  Pubblicazione: (2025)
• Difference Rewards Policy Gradients
  di: Castellini, Jacopo, et al.
  Pubblicazione: (2020)