Image de couverture de livre

Memory-Efficient Optimization with Factorized Hamiltonian Descent

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Nguyen, Son, Chen, Lizhang, Liu, Bo, Liu, Qiang
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Memory-Efficient LLM Training with Online Subspace Descent
par: Liang, Kaizhao, et autres
Publié: (2024)

Improving Adaptive Moment Optimization via Preconditioner Diagonalization
par: Nguyen, Son, et autres
Publié: (2025)

Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts
par: Chen, Lizhang, et autres
Publié: (2023)

Muon Optimizes Under Spectral Norm Constraints
par: Chen, Lizhang, et autres
Publié: (2025)

Cautious Optimizers: Improving Training with One Line of Code
par: Liang, Kaizhao, et autres
Publié: (2024)

Hamiltonian Descent Algorithms for Optimization: Accelerated Rates via Randomized Integration Time
par: Fu, Qiang, et autres
Publié: (2025)

DeMo: Decoupled Momentum Optimization
par: Peng, Bowen, et autres
Publié: (2024)

Training-Free Looped Transformers
par: Chen, Lizhang, et autres
Publié: (2026)

Communication Efficient Distributed Training with Distributed Lion
par: Liu, Bo, et autres
Publié: (2024)

Conflict-Averse Gradient Descent for Multi-task Learning
par: Liu, Bo, et autres
Publié: (2021)

Preconditioning for Accelerated Gradient Descent Optimization and Regularization
par: Ye, Qiang
Publié: (2024)

Omni-Masked Gradient Descent: Memory-Efficient Optimization via Mask Traversal with Improved Convergence
par: Yang, Hui, et autres
Publié: (2026)

Quantum Optimization via Gradient-Based Hamiltonian Descent
par: Leng, Jiaqi, et autres
Publié: (2025)

Momentum Guidance: Plug-and-Play Guidance for Flow Models
par: Liao, Runlong, et autres
Publié: (2026)

When Descent Is Too Stable: Event-Triggered Hamiltonian Learning to Optimize
par: Wang, Yi, et autres
Publié: (2026)

Hidden State Differential Private Mini-Batch Block Coordinate Descent for Multi-convexity Optimization
par: Chen, Ding, et autres
Publié: (2024)

FluidML: Fast and Memory Efficient Inference Optimization
par: Liu, Jinjie, et autres
Publié: (2024)

Scalable Community Detection Using Quantum Hamiltonian Descent and QUBO Formulation
par: Cheng, Jinglei, et autres
Publié: (2024)

$ϕ$-Balancing for Mixture-of-Experts Training
par: Chen, Lizhang, et autres
Publié: (2026)

Multiple Wasserstein Gradient Descent Algorithm for Multi-Objective Distributional Optimization
par: Nguyen, Dai Hai, et autres
Publié: (2025)

COSMOS: A Hybrid Adaptive Optimizer for Memory-Efficient Training of LLMs
par: Liu, Liming, et autres
Publié: (2025)

Revisiting the Initial Steps in Adaptive Gradient Descent Optimization
par: Abuduweili, Abulikemu, et autres
Publié: (2024)

A Memory Efficient Randomized Subspace Optimization Method for Training Large Language Models
par: Chen, Yiming, et autres
Publié: (2025)

SMMF: Square-Matricized Momentum Factorization for Memory-Efficient Optimization
par: Park, Kwangryeol, et autres
Publié: (2024)

CUPID in the Model Zoo: Online Matchmaking for Selecting Your Dream LLM
par: Nguyen, Son, et autres
Publié: (2026)

Optimizing Canaries for Privacy Auditing with Metagradient Descent
par: Boglioni, Matteo, et autres
Publié: (2025)

Learning Associative Memories with Gradient Descent
par: Cabannes, Vivien, et autres
Publié: (2024)

Cautious Weight Decay
par: Chen, Lizhang, et autres
Publié: (2025)

Self-Auditing Parameter-Efficient Fine-Tuning for Few-Shot 3D Medical Image Segmentation
par: Ly, Son Thai, et autres
Publié: (2026)

Efficient Deep Learning for Medical Imaging: Bridging the Gap Between High-Performance AI and Clinical Deployment
par: Nguyen, Cuong Manh, et autres
Publié: (2026)

One-Step Flow Policy Mirror Descent
par: Chen, Tianyi, et autres
Publié: (2025)

Optimizing ML Training with Metagradient Descent
par: Engstrom, Logan, et autres
Publié: (2025)

FedBCD:Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learning
par: Liu, Junkang, et autres
Publié: (2026)

PowerStep: Memory-Efficient Adaptive Optimization via $\ell_p$-Norm Steepest Descent
par: Lu, Yao, et autres
Publié: (2026)

Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent
par: Chen, Bo, et autres
Publié: (2024)

Exploring the Optimized Value of Each Hyperparameter in Various Gradient Descent Algorithms
par: Chen, Abel C. H.
Publié: (2022)

Not All Adapters Matter: Selective Adapter Freezing for Memory-Efficient Fine-Tuning of Language Models
par: Son, Hyegang, et autres
Publié: (2024)

Low-Tubal-Rank Tensor Recovery via Factorized Gradient Descent
par: Liu, Zhiyu, et autres
Publié: (2024)

Low-rank Momentum Factorization for Memory Efficient Training
par: Mahdavinia, Pouria, et autres
Publié: (2025)

Beyond Cross-Validation: Adaptive Parameter Selection for Kernel-Based Gradient Descents
par: Liu, Xiaotong, et autres
Publié: (2026)