Cover Image

SMMF: Square-Matricized Momentum Factorization for Memory-Efficient Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Park, Kwangryeol, Lee, Seulki
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TransPL: VQ-Code Transition Matrices for Pseudo-Labeling of Time Series Unsupervised Domain Adaptation
by: Kim, Jaeho, et al.
Published: (2025)

CAFO: Feature-Centric Explanation on Time Series Classification
by: Kim, Jaeho, et al.
Published: (2024)

DeMo: Decoupled Momentum Optimization
by: Peng, Bowen, et al.
Published: (2024)

On the Limits of Momentum in Decentralized and Federated Optimization
by: Zaccone, Riccardo, et al.
Published: (2025)

FlashOptim: Optimizers for Memory-Efficient Training
by: Ortiz, Jose Javier Gonzalez, et al.
Published: (2026)

BLAST: Block-Level Adaptive Structured Matrices for Efficient Deep Neural Network Inference
by: Lee, Changwoo, et al.
Published: (2024)

Momentum Boosted Episodic Memory for Improving Learning in Long-Tailed RL Environments
by: Fernandes, Dolton, et al.
Published: (2025)

LOOKAT: Lookup-Optimized Key-Attention for Memory-Efficient Transformers
by: Karmore, Aryan
Published: (2026)

LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices
by: Lee, Jung Hyun, et al.
Published: (2024)

Optimization Guarantees for Square-Root Natural-Gradient Variational Inference
by: Kumar, Navish, et al.
Published: (2025)

Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization
by: Shen, Qianli, et al.
Published: (2024)

MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
by: Liu, Yuxi, et al.
Published: (2025)

Pretraining a Shared Q-Network for Data-Efficient Offline Reinforcement Learning
by: Park, Jongchan, et al.
Published: (2025)

TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizers
by: Cheng, Peng, et al.
Published: (2026)

Momentum Streams for Optimizer-Inspired Transformers
by: Gai, Jingchu, et al.
Published: (2026)

KernelSHAP-IQ: Weighted Least-Square Optimization for Shapley Interactions
by: Fumagalli, Fabian, et al.
Published: (2024)

Wonderful Matrices: More Efficient and Effective Architecture for Language Modeling Tasks
by: Shi, Jingze, et al.
Published: (2024)

COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training
by: Xi, Haocheng, et al.
Published: (2024)

MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
by: Chen, Yupeng, et al.
Published: (2024)

How to Square Tensor Networks and Circuits Without Squaring Them
by: Loconte, Lorenzo, et al.
Published: (2025)

Resource-Efficient Transformer Architecture: Optimizing Memory and Execution Time for Real-Time Applications
by: V, Krisvarish, et al.
Published: (2024)

CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimization
by: Yang, Zi, et al.
Published: (2024)

Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers
by: Ranganath, Aditya
Published: (2026)

Torque-Aware Momentum
by: Malviya, Pranshu, et al.
Published: (2024)

Excitation: Momentum For Experts
by: Shaier, Sagi
Published: (2026)

FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuning
by: Yan, Peishen, et al.
Published: (2026)

Riemannian Optimization for LoRA on the Stiefel Manifold
by: Park, Juneyoung, et al.
Published: (2025)

Tri-Accel: Curvature-Aware Precision-Adaptive and Memory-Elastic Optimization for Efficient GPU Usage
by: Sheibanian, Mohsen, et al.
Published: (2025)

SNOO: Step-K Nesterov Outer Optimizer - The Surprising Effectiveness of Nesterov Momentum Applied to Pseudo-Gradients
by: Kallusky, Dominik, et al.
Published: (2025)

Inversion-based Latent Bayesian Optimization
by: Chu, Jaewon, et al.
Published: (2024)

Lattice: Learning to Efficiently Compress the Memory
by: Karami, Mahdi, et al.
Published: (2025)

Memory-Efficient LLM Pretraining via Minimalist Optimizer Design
by: Glentis, Athanasios, et al.
Published: (2025)

Memory-Efficient Acceleration of Block Low-Rank Foundation Models on Resource Constrained GPUs
by: Abillama, Pierre, et al.
Published: (2025)

OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization
by: Boss, Mark, et al.
Published: (2026)

Double Momentum Method for Lower-Level Constrained Bilevel Optimization
by: Shi, Wanli, et al.
Published: (2024)

Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimization
by: Yun, Taeyoung, et al.
Published: (2024)

OvA-LP: A Simple and Efficient Framework for Federated Learning on Non-IID Data
by: Park, Dongjin, et al.
Published: (2025)

Semi-Supervised Preference Optimization with Limited Feedback
by: Lee, Seonggyun, et al.
Published: (2025)

SketchOGD: Memory-Efficient Continual Learning
by: Min, Youngjae, et al.
Published: (2023)

DUEL: Duplicate Elimination on Active Memory for Self-Supervised Class-Imbalanced Learning
by: Choi, Won-Seok, et al.
Published: (2024)