Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models

Documenti analoghi

• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations
  di: Ma, Shaocong, et al.
  Pubblicazione: (2025)
• LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
  di: Pan, Rui, et al.
  Pubblicazione: (2024)
• Zeroth-Order Optimization Finds Flat Minima
  di: Zhang, Liang, et al.
  Pubblicazione: (2025)
• On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization
  di: Ma, Shaocong, et al.
  Pubblicazione: (2025)
• Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective
  di: Fernando, Heshan, et al.
  Pubblicazione: (2024)