Cover Image

Modified Loss of Momentum Gradient Descent: Fine-Grained Analysis

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Cattaneo, Matias D., Shigida, Boris
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Numerical Analysis Optimization and Control Computation
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

How Memory in Optimization Algorithms Implicitly Modifies the Loss
by: Cattaneo, Matias D., et al.
Published: (2025)

The Effect of Mini-Batch Noise on the Implicit Bias of Adam
by: Cattaneo, Matias D., et al.
Published: (2026)

Policy Gradient with Second Order Momentum
by: Sun, Tianyu
Published: (2025)

Super Gradient Descent: Global Optimization requires Global Gradient
by: Achour, Seifeddine
Published: (2024)

Provably Faster Gradient Descent via Long Steps
by: Grimmer, Benjamin
Published: (2023)

Gradient Descent as a Perceptron Algorithm: Understanding Dynamics and Implicit Acceleration
by: Tyurin, Alexander
Published: (2025)

Gradient is All You Need? How Consensus-Based Optimization can be Interpreted as a Stochastic Relaxation of Gradient Descent
by: Riedl, Konstantin, et al.
Published: (2023)

Convergence Analysis of Fractional Gradient Descent
by: Aggarwal, Ashwani
Published: (2023)

Fast and Provable Tensor-Train Format Tensor Completion via Precondtioned Riemannian Gradient Descent
by: Bian, Fengmiao, et al.
Published: (2025)

Dual Cone Gradient Descent for Training Physics-Informed Neural Networks
by: Hwang, Youngsik, et al.
Published: (2024)

On the Convergence of the Gradient Descent Method with Stochastic Fixed-point Rounding Errors under the Polyak-Lojasiewicz Inequality
by: Xia, Lu, et al.
Published: (2023)

On the Implicit Bias of Adam
by: Cattaneo, Matias D., et al.
Published: (2023)

Quantitative Convergences of Lie Group Momentum Optimizers
by: Kong, Lingkai, et al.
Published: (2024)

Block Acceleration Without Momentum: On Optimal Stepsizes of Block Gradient Descent for Least-Squares
by: Peng, Liangzu, et al.
Published: (2024)

A Family of Controllable Momentum Coefficients for Forward-Backward Accelerated Algorithms
by: Fu, Mingwei, et al.
Published: (2025)

A Block Coordinate Descent Method for Nonsmooth Composite Optimization under Orthogonality Constraints
by: Yuan, Ganzhao
Published: (2023)

A Single-Loop Gradient Descent and Perturbed Ascent Algorithm for Nonconvex Functional Constrained Optimization
by: Lu, Songtao
Published: (2022)

Faster Randomized Methods for Orthogonality Constrained Problems
by: Shustin, Boris, et al.
Published: (2021)

Adaptive Proximal Gradient Method for Convex Optimization
by: Malitsky, Yura, et al.
Published: (2023)

Enhanced Adaptive Gradient Algorithms for Nonconvex-PL Minimax Optimization
by: Huang, Feihu, et al.
Published: (2023)

Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models
by: Zhang, Fangzhao, et al.
Published: (2024)

Dynamic Proximal Gradient Algorithms for Schatten-$p$ Quasi-Norm Regularized Problems
by: Shen, Weiping, et al.
Published: (2026)

Fast Unconstrained Optimization via Hessian Averaging and Adaptive Gradient Sampling Methods
by: O'Leary-Roseberry, Thomas, et al.
Published: (2024)

Adaptive Lipschitz-Free Conditional Gradient Methods for Stochastic Composite Nonconvex Optimization
by: Yuan, Ganzhao
Published: (2026)

A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equations
by: Liu, Shu, et al.
Published: (2024)

PowerStep: Memory-Efficient Adaptive Optimization via $\ell_p$-Norm Steepest Descent
by: Lu, Yao, et al.
Published: (2026)

Low-Discrepancy Set Post-Processing via Gradient Descent
by: Clément, François, et al.
Published: (2025)

IRKA is a Riemannian Gradient Descent Method
by: Mlinarić, Petar, et al.
Published: (2023)

Lyapunov Analysis For Monotonically Forward-Backward Accelerated Algorithms
by: Fu, Mingwei, et al.
Published: (2024)

Online Covariance Matrix Estimation in Sketched Newton Methods
by: Kuang, Wei, et al.
Published: (2025)

Bayesian Optimization on Networks
by: Li, Wenwen, et al.
Published: (2025)

Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models
by: Fang, Yuchen, et al.
Published: (2024)

The Essential Best and Average Rate of Convergence of the Exact Line Search Gradient Descent Method
by: Yu, Thomas
Published: (2023)

Fine-grained Analysis and Faster Algorithms for Iteratively Solving Linear Systems
by: Dereziński, Michał, et al.
Published: (2024)

Generative Neural Operators of Log-Complexity Can Simultaneously Solve Infinitely Many Convex Programs
by: Kratsios, Anastasis, et al.
Published: (2025)

A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learning
by: Toscano, Juan Diego, et al.
Published: (2025)

High Probability Complexity Bounds of Trust-Region Stochastic Sequential Quadratic Programming with Heavy-Tailed Noise
by: Fang, Yuchen, et al.
Published: (2025)

A single-loop SPIDER-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth optimization
by: Liu, Wei, et al.
Published: (2025)

Faster Computation of Entropic Optimal Transport via Stable Low Frequency Modes
by: Chhaibi, Reda, et al.
Published: (2025)

Stability Preserving Data-driven Models With Latent Dynamics
by: Luo, Yushuang, et al.
Published: (2022)