Greedy-Gnorm: A Gradient Matrix Norm-Based Alternative to Attention Entropy for Head Pruning

Documenti analoghi

• Efficient LLMs with AMP: Attention Heads and MLP Pruning
  di: Mugnaini, Leandro Giusti, et al.
  Pubblicazione: (2025)
• Rényi Attention Entropy for Patch Pruning
  di: Aizawa, Hiroaki, et al.
  Pubblicazione: (2026)
• Convergence of Alternating Gradient Descent for Matrix Factorization
  di: Ward, Rachel, et al.
  Pubblicazione: (2023)
• Attention Head Entropy of LLMs Predicts Answer Correctness
  di: Ostmeier, Sophie, et al.
  Pubblicazione: (2026)
• Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters
  di: Yukhimchuk, Alexander, et al.
  Pubblicazione: (2026)