Cover Image

Gradient Descent as a Perceptron Algorithm: Understanding Dynamics and Implicit Acceleration

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Tyurin, Alexander
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Numerical Analysis Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Super Gradient Descent: Global Optimization requires Global Gradient
by: Achour, Seifeddine
Published: (2024)

Provably Faster Gradient Descent via Long Steps
by: Grimmer, Benjamin
Published: (2023)

Gradient is All You Need? How Consensus-Based Optimization can be Interpreted as a Stochastic Relaxation of Gradient Descent
by: Riedl, Konstantin, et al.
Published: (2023)

Dynamic Proximal Gradient Algorithms for Schatten-$p$ Quasi-Norm Regularized Problems
by: Shen, Weiping, et al.
Published: (2026)

Fast and Provable Tensor-Train Format Tensor Completion via Precondtioned Riemannian Gradient Descent
by: Bian, Fengmiao, et al.
Published: (2025)

Dual Cone Gradient Descent for Training Physics-Informed Neural Networks
by: Hwang, Youngsik, et al.
Published: (2024)

On the Convergence of the Gradient Descent Method with Stochastic Fixed-point Rounding Errors under the Polyak-Lojasiewicz Inequality
by: Xia, Lu, et al.
Published: (2023)

Modified Loss of Momentum Gradient Descent: Fine-Grained Analysis
by: Cattaneo, Matias D., et al.
Published: (2025)

A Single-Loop Gradient Descent and Perturbed Ascent Algorithm for Nonconvex Functional Constrained Optimization
by: Lu, Songtao
Published: (2022)

Lyapunov Analysis For Monotonically Forward-Backward Accelerated Algorithms
by: Fu, Mingwei, et al.
Published: (2024)

Enhanced Adaptive Gradient Algorithms for Nonconvex-PL Minimax Optimization
by: Huang, Feihu, et al.
Published: (2023)

A Family of Controllable Momentum Coefficients for Forward-Backward Accelerated Algorithms
by: Fu, Mingwei, et al.
Published: (2025)

Convergence Analysis of Fractional Gradient Descent
by: Aggarwal, Ashwani
Published: (2023)

Near-Optimal Convergence of Accelerated Gradient Methods under Generalized and $(L_0, L_1)$-Smoothness
by: Tyurin, Alexander
Published: (2025)

A Block Coordinate Descent Method for Nonsmooth Composite Optimization under Orthogonality Constraints
by: Yuan, Ganzhao
Published: (2023)

Acceleration Methods
by: d'Aspremont, Alexandre, et al.
Published: (2021)

Policy Gradient with Second Order Momentum
by: Sun, Tianyu
Published: (2025)

Block Acceleration Without Momentum: On Optimal Stepsizes of Block Gradient Descent for Least-Squares
by: Peng, Liangzu, et al.
Published: (2024)

Adaptive Proximal Gradient Method for Convex Optimization
by: Malitsky, Yura, et al.
Published: (2023)

Scalable Acceleration for Classification-Based Derivative-Free Optimization
by: Han, Tianyi, et al.
Published: (2023)

Fast Unconstrained Optimization via Hessian Averaging and Adaptive Gradient Sampling Methods
by: O'Leary-Roseberry, Thomas, et al.
Published: (2024)

Adaptive Lipschitz-Free Conditional Gradient Methods for Stochastic Composite Nonconvex Optimization
by: Yuan, Ganzhao
Published: (2026)

Anderson Acceleration in Nonsmooth Problems: Local Convergence via Active Manifold Identification
by: Li, Kexin, et al.
Published: (2024)

WinQ: Accelerating Quantization-Aware Training of Language Models Around Saddle Points
by: Li, Dongyue, et al.
Published: (2026)

A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equations
by: Liu, Shu, et al.
Published: (2024)

Worth Their Weight: Randomized and Regularized Block Kaczmarz Algorithms without Preprocessing
by: Goldshlager, Gil, et al.
Published: (2025)

Efficient Algorithms for Regularized Nonnegative Scale-invariant Low-rank Approximation Models
by: Cohen, Jeremy E., et al.
Published: (2024)

UAdam: Unified Adam-Type Algorithmic Framework for Non-Convex Stochastic Optimization
by: Jiang, Yiming, et al.
Published: (2023)

IRKA is a Riemannian Gradient Descent Method
by: Mlinarić, Petar, et al.
Published: (2023)

Flattened one-bit stochastic gradient descent: compressed distributed optimization with controlled variance
by: Stollenwerk, Alexander, et al.
Published: (2024)

Low-Discrepancy Set Post-Processing via Gradient Descent
by: Clément, François, et al.
Published: (2025)

On the Stability Connection Between Discrete-Time Algorithms and Their Resolution ODEs: Applications to Min-Max Optimisation
by: Farzin, Amir Ali, et al.
Published: (2026)

Using Linearized Optimal Transport to Predict the Evolution of Stochastic Particle Systems
by: Karris, Nicholas, et al.
Published: (2024)

The Essential Best and Average Rate of Convergence of the Exact Line Search Gradient Descent Method
by: Yu, Thomas
Published: (2023)

Learning Regularization Functionals for Inverse Problems: A Comparative Study
by: Hertrich, Johannes, et al.
Published: (2025)

On the numerical reliability of nonsmooth autodiff: a MaxPool case study
by: Boustany, Ryan
Published: (2024)

Continuum-marginal optimal transport: a mesh-free kernel method
by: Nakano, Yumiharu
Published: (2026)

A Discrete Variational Derivation of Accelerated Methods in Optimization
by: Campos, Cédric M., et al.
Published: (2021)

End-to-End Mesh Optimization of a Hybrid Deep Learning Black-Box PDE Solver
by: Ma, Shaocong, et al.
Published: (2024)

Suboptimality bounds for trace-bounded SDPs enable a faster and scalable low-rank SDP solver SDPLR+
by: Huang, Yufan, et al.
Published: (2024)