Train Less, Infer Faster: Efficient Model Finetuning and Compression via Structured Sparsity

Documenti analoghi

• FineGates: LLMs Finetuning with Compression using Stochastic Gates
  di: Svirsky, Jonathan, et al.
  Pubblicazione: (2024)
• AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning
  di: Refael, Yehonathan, et al.
  Pubblicazione: (2024)
• TransformLLM: Adapting Large Language Models via LLM-Transformed Reading Comprehension Text
  di: Arbel, Iftach, et al.
  Pubblicazione: (2024)
• LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM
  di: Refael, Yehonathan, et al.
  Pubblicazione: (2025)
• SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training
  di: Refael, Yehonathan, et al.
  Pubblicazione: (2025)