FFT-based Dynamic Subspace Selection for Low-Rank Adaptive Optimization of Large Language Models

Documenti analoghi

• MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning
  di: Modoranu, Ionut-Vlad, et al.
  Pubblicazione: (2026)
• LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics
  di: Robert, Thomas, et al.
  Pubblicazione: (2024)
• DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers
  di: Modoranu, Ionut-Vlad, et al.
  Pubblicazione: (2026)
• MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence
  di: Modoranu, Ionut-Vlad, et al.
  Pubblicazione: (2024)
• The Iterative Optimal Brain Surgeon: Faster Sparse Recovery by Leveraging Second-Order Information
  di: Wu, Diyuan, et al.
  Pubblicazione: (2024)