Copertina

Low-rank Momentum Factorization for Memory Efficient Training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Mahdavinia, Pouria, Mahdavi, Mehrdad
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Harnessing Optimization Dynamics for Curvature-Informed Model Merging
di: Mahdavinia, Pouria, et al.
Pubblicazione: (2025)

On the Convergence and Stability of Distributed Sub-model Training
di: Deng, Yuyang, et al.
Pubblicazione: (2025)

MLorc: Momentum Low-rank Compression for Memory Efficient Large Language Model Adaptation
di: Shen, Wei, et al.
Pubblicazione: (2025)

Merge before Forget: A Single LoRA Continual Learning via Continual Merging
di: Qiao, Fuli, et al.
Pubblicazione: (2025)

RefGrader: Automated Grading of Mathematical Competition Proofs using Agentic Workflows
di: Mahdavi, Hamed, et al.
Pubblicazione: (2025)

Model Merging via Multi-Teacher Knowledge Distillation
di: Dalili, Seyed Arshan, et al.
Pubblicazione: (2025)

SMMF: Square-Matricized Momentum Factorization for Memory-Efficient Optimization
di: Park, Kwangryeol, et al.
Pubblicazione: (2024)

Stochastic Compositional Minimax Optimization with Provable Convergence Guarantees
di: Deng, Yuyang, et al.
Pubblicazione: (2024)

On the Generalization Ability of Unsupervised Pretraining
di: Deng, Yuyang, et al.
Pubblicazione: (2024)

Gradient Weight-normalized Low-rank Projection for Efficient LLM Training
di: Huang, Jia-Hong, et al.
Pubblicazione: (2024)

On the Generalization Capability of Temporal Graph Learning Algorithms: Theoretical Insights and a Simpler Method
di: Cong, Weilin, et al.
Pubblicazione: (2024)

Alada: Alternating Adaptation of Momentum Method for Memory-Efficient Matrix Optimization
di: He, Xiaoyu, et al.
Pubblicazione: (2025)

Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients
di: Wang, Yezhen, et al.
Pubblicazione: (2025)

PLUMAGE: Probabilistic Low rank Unbiased Min Variance Gradient Estimator for Efficient Large Model Training
di: Haroush, Matan, et al.
Pubblicazione: (2025)

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
di: Zhao, Jiawei, et al.
Pubblicazione: (2024)

Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients
di: Muhamed, Aashiq, et al.
Pubblicazione: (2024)

Quantum Speedups for Markov Chain Monte Carlo Methods with Application to Optimization
di: Ozgul, Guneykan, et al.
Pubblicazione: (2025)

Loki: Low-rank Keys for Efficient Sparse Attention
di: Singhania, Prajwal, et al.
Pubblicazione: (2024)

Understanding Deep Gradient Leakage via Inversion Influence Functions
di: Zhang, Haobo, et al.
Pubblicazione: (2023)

Training-Conditional Coverage Bounds under Covariate Shift
di: Pournaderi, Mehrdad, et al.
Pubblicazione: (2024)

Training-Conditional Coverage Bounds for Uniformly Stable Learning Algorithms
di: Pournaderi, Mehrdad, et al.
Pubblicazione: (2024)

Robust Low-rank Tensor Train Recovery
di: Qin, Zhen, et al.
Pubblicazione: (2024)

Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?
di: Chen, Xi, et al.
Pubblicazione: (2024)

Distributed Low-Communication Training with Decoupled Momentum Optimization
di: Nedelkoski, Sasho, et al.
Pubblicazione: (2025)

Recycling History: Efficient Recommendations from Contextual Dueling Bandits
di: Sankagiri, Suryanarayana, et al.
Pubblicazione: (2025)

Efficient Differentiable Approximation of Generalized Low-rank Regularization
di: Li, Naiqi, et al.
Pubblicazione: (2025)

Efficient Duple Perturbation Robustness in Low-rank MDPs
di: Hu, Yang, et al.
Pubblicazione: (2024)

Fourier Low-rank and Sparse Tensor for Efficient Tensor Completion
di: Li, Jingyang, et al.
Pubblicazione: (2025)

Memory-Efficient Optimization with Factorized Hamiltonian Descent
di: Nguyen, Son, et al.
Pubblicazione: (2024)

DeepCover: Advancing RNN Test Coverage and Online Error Prediction using State Machine Extraction
di: Golshanrad, Pouria, et al.
Pubblicazione: (2024)

Leveraging Low-rank Factorizations of Conditional Correlation Matrices in Graph Learning
di: Phi, Thu Ha, et al.
Pubblicazione: (2025)

Bridging Training and Merging Through Momentum-Aware Optimization
di: Moayedikia, Alireza, et al.
Pubblicazione: (2025)

Distributed Sign Momentum with Local Steps for Training Transformers
di: Yu, Shuhua, et al.
Pubblicazione: (2024)

FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuning
di: Yan, Peishen, et al.
Pubblicazione: (2026)

Dynamically Weighted Momentum with Adaptive Step Sizes for Efficient Deep Network Training
di: Wang, Zhifeng, et al.
Pubblicazione: (2025)

Task-agnostic Low-rank Residual Adaptation for Efficient Federated Continual Fine-Tuning
di: Yu, Feng, et al.
Pubblicazione: (2025)

A Semi Centralized Training Decentralized Execution Architecture for Multi Agent Deep Reinforcement Learning in Traffic Signal Control
di: Rezaali, Arash, et al.
Pubblicazione: (2025)

OASIS: Online Activation Subspace Learning for Memory-Efficient Training
di: Choudhary, Sakshi, et al.
Pubblicazione: (2026)

A New Approach to Backtracking Counterfactual Explanations: A Unified Causal Framework for Efficient Model Interpretability
di: Fatemi, Pouria, et al.
Pubblicazione: (2025)

LRD-MPC: Efficient MPC Inference through Low-rank Decomposition
di: Tang, Tingting, et al.
Pubblicazione: (2026)