Image de couverture de livre

RMNP: Row-Momentum Normalized Preconditioning for Scalable Matrix-Based Optimization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Deng, Shenyang, Ouyang, Zhuoli, Pang, Tianyu, Liu, Zihang, Jin, Ruochen, Yu, Shuhua, Yang, Yaoqing
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Depth, Not Data: An Analysis of Hessian Spectral Bifurcation
par: Deng, Shenyang, et autres
Publié: (2026)

Suspicious Alignment of SGD: A Fine-Grained Step Size Condition Analysis
par: Deng, Shenyang, et autres
Publié: (2026)

HTMuon: Improving Muon via Heavy-Tailed Spectral Correction
par: Pang, Tianyu, et autres
Publié: (2026)

From Spikes to Heavy Tails: Unveiling the Spectral Evolution of Neural Networks
par: Kothapalli, Vignesh, et autres
Publié: (2024)

Nora: Normalized Orthogonal Row Alignment for Scalable Matrix Optimizer
par: Yuan, Jinghui, et autres
Publié: (2026)

Balancing Learning Rates Across Layers: Exact Two-Step Dynamics and Optimal Scaling in Linear Neural Networks
par: Pang, Tianyu, et autres
Publié: (2026)

Model Balancing Helps Low-data Training and Fine-tuning
par: Liu, Zihang, et autres
Publié: (2024)

KCES: Training-Free Defense for Robust Graph Neural Networks via Kernel Complexity
par: Jia, Yaning, et autres
Publié: (2025)

LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning
par: Liu, Zihang, et autres
Publié: (2025)

RL4RLA: Teaching ML to Discover Randomized Linear Algebra Algorithms Through Curriculum Design and Graph-Based Search
par: Xiong, Jinglong, et autres
Publié: (2026)

Learning to Discover Iterative Spectral Algorithms
par: Liu, Zihang, et autres
Publié: (2026)

Alada: Alternating Adaptation of Momentum Method for Memory-Efficient Matrix Optimization
par: He, Xiaoyu, et autres
Publié: (2025)

Scalable Mean-Field Variational Inference via Preconditioned Primal-Dual Optimization
par: Lyu, Jinhua, et autres
Publié: (2026)

Distributed Sign Momentum with Local Steps for Training Transformers
par: Yu, Shuhua, et autres
Publié: (2024)

Optimizing the Adversarial Perturbation with a Momentum-based Adaptive Matrix
par: Tao, Wei, et autres
Publié: (2025)

On the Width Scaling of Neural Optimizers Under Matrix Operator Norms I: Row/Column Normalization and Hyperparameter Transfer
par: Xu, Ruihan, et autres
Publié: (2026)

Hyperparameter Transfer Enables Consistent Gains of Matrix-Preconditioned Optimizers Across Scales
par: Qiu, Shikai, et autres
Publié: (2025)

SGD with Adaptive Preconditioning: Unified Analysis and Momentum Acceleration
par: Kovalev, Dmitry
Publié: (2025)

Understanding Generalization from Embedding Dimension and Distributional Convergence
par: Yu, Junjie, et autres
Publié: (2026)

Adaptive Optimization via Momentum on Variance-Normalized Gradients
par: Patitucci, Francisco, et autres
Publié: (2026)

PROMISE: Preconditioned Stochastic Optimization Methods by Incorporating Scalable Curvature Estimates
par: Frangella, Zachary, et autres
Publié: (2023)

DP-Muon: Differentially Private Optimization via Matrix-Orthogonalized Momentum
par: Kim, Jihwan, et autres
Publié: (2026)

AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix
par: Yue, Yun, et autres
Publié: (2023)

Muown: Row-Norm Control for Muon Optimization
par: Lion, Kai, et autres
Publié: (2026)

PolarGrad: A Class of Matrix-Gradient Optimizers from a Unifying Preconditioning Perspective
par: Lau, Tim Tsz-Kit, et autres
Publié: (2025)

Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets
par: Hsiung, Lei, et autres
Publié: (2025)

Policy Gradient with Second Order Momentum
par: Sun, Tianyu
Publié: (2025)

Unveiling Multi-regime Patterns in SciML: Distinct Failure Modes and Regime-specific Optimization
par: Wang, Yuxin, et autres
Publié: (2026)

Preconditioned Gradient Descent for Over-Parameterized Nonconvex Matrix Factorization
par: Zhang, Gavin, et autres
Publié: (2025)

NysAct: A Scalable Preconditioned Gradient Descent using Nystrom Approximation
par: Seung, Hyunseok, et autres
Publié: (2025)

The Power of Preconditioning in Overparameterized Low-Rank Matrix Sensing
par: Xu, Xingyu, et autres
Publié: (2023)

Stochastic Normalized Gradient Descent with Momentum for Large-Batch Training
par: Zhao, Shen-Yi, et autres
Publié: (2020)

Spectral Insights into Data-Oblivious Critical Layers in Large Language Models
par: Liu, Xuyuan, et autres
Publié: (2025)

REFORMER: A ChatGPT-Driven Data Synthesis Framework Elevating Text-to-SQL Models
par: Liu, Shenyang, et autres
Publié: (2025)

Preconditioning for Accelerated Gradient Descent Optimization and Regularization
par: Ye, Qiang
Publié: (2024)

Efficient Over-parameterized Matrix Sensing from Noisy Measurements via Alternating Preconditioned Gradient Descent
par: Liu, Zhiyu, et autres
Publié: (2025)

A Unifying View of Linear Function Approximation in Off-Policy RL Through Matrix Splitting and Preconditioning
par: Wu, Zechen, et autres
Publié: (2025)

Error Feedback under $(L_0,L_1)$-Smoothness: Normalization and Momentum
par: Khirirat, Sarit, et autres
Publié: (2024)

Guarantees of a Preconditioned Subgradient Algorithm for Overparameterized Asymmetric Low-rank Matrix Recovery
par: Giampouras, Paris, et autres
Publié: (2024)

A Three-regime Model of Network Pruning
par: Zhou, Yefan, et autres
Publié: (2023)