Cover Image

Dynamics of SGD with Stochastic Polyak Stepsizes: Truly Adaptive Variants and Convergence to Exact Solution

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Orvieto, Antonio, Lacoste-Julien, Simon, Loizou, Nicolas
Format:	Preprint
Published:	2022
Subjects:	Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Adaptive SGD with Line-Search and Polyak Stepsizes: Nonconvex Convergence and Accelerated Rates
by: Wu, Haotian
Published: (2025)

Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance
by: Oikonomou, Dimitris, et al.
Published: (2024)

An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes
by: Orvieto, Antonio, et al.
Published: (2024)

Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients
by: Oikonomou, Dimitris, et al.
Published: (2025)

Adaptive Sharpness-Aware Minimization with a Polyak-type Step size: A Theory-Grounded Scheduler
by: Oikonomou, Dimitris, et al.
Published: (2026)

Adaptive Polyak Stepsize with Level-value Adjustment for Distributed Optimization
by: Ouyang, Chen, et al.
Published: (2026)

New Results on the Polyak Stepsize: Tight Convergence Analysis and Universal Function Classes
by: He, Chang, et al.
Published: (2025)

Accelerating Level-Value Adjustment for the Polyak Stepsize
by: Liu, Anbang, et al.
Published: (2023)

Cutting Some Slack for SGD with Adaptive Polyak Stepsizes
by: Gower, Robert M., et al.
Published: (2022)

Exact Convergence rate of the subgradient method by using Polyak step size
by: Zamani, Moslem, et al.
Published: (2024)

Polyak Stepsize: Estimating Optimal Functional Values Without Parameters or Prior Knowledge
by: Abdukhakimov, Farshed, et al.
Published: (2025)

New Perspectives on the Polyak Stepsize: Surrogate Functions and Negative Results
by: Orabona, Francesco, et al.
Published: (2025)

Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsize
by: Li, Chris Junchi
Published: (2024)

Is your batch size the problem? Revisiting the Adam-SGD gap in language modeling
by: Srećković, Teodora, et al.
Published: (2025)

Stochastic Extragradient with Random Reshuffling: Improved Convergence for Variational Inequalities
by: Emmanouilidis, Konstantinos, et al.
Published: (2024)

Linear Convergence of the Proximal Gradient Method for Composite Optimization Under the Polyak-Łojasiewicz Inequality and Its Variant
by: Kong, Qingyuan, et al.
Published: (2024)

Coupling-based Convergence Diagnostic and Stepsize Scheme for Stochastic Gradient Descent
by: Li, Xiang, et al.
Published: (2024)

Acceleration for Polyak-Łojasiewicz Functions with a Gradient Aiming Condition
by: Hermant, Julien
Published: (2026)

High-Probability Bounds for SGD under the Polyak-Lojasiewicz Condition with Markovian Noise
by: Kar, Avik, et al.
Published: (2026)

Generalization of Silver Stepsize Schedule to Stochastic Optimization
by: Bai, Luwei, et al.
Published: (2025)

Stochastic Block Bregman Projection with Polyak-like Stepsize for Possibly Inconsistent Convex Feasibility Problems
by: Zhang, Lu, et al.
Published: (2026)

Adaptive Stepsize Selection in Decentralized Convex Optimization
by: Kuruzov, Ilya, et al.
Published: (2025)

Dual Optimistic Ascent (PI Control) is the Augmented Lagrangian Method in Disguise
by: Ramirez, Juan, et al.
Published: (2025)

On the Convergence of DP-SGD with Adaptive Clipping
by: Shulgin, Egor, et al.
Published: (2024)

Locally Adaptive Federated Learning
by: Mukherjee, Sohom, et al.
Published: (2023)

Recurrent neural networks: vanishing and exploding gradients are not the end of the story
by: Zucchet, Nicolas, et al.
Published: (2024)

Stochastic Approximation with Block Coordinate Optimal Stepsizes
by: Jiang, Tao, et al.
Published: (2025)

Revisiting the Constant Stepsize Stochastic Approximation with Decision-Dependent Markovian Noise
by: Hadavi, Hadi, et al.
Published: (2026)

Non-convex Stochastic Composite Optimization with Polyak Momentum
by: Gao, Yuan, et al.
Published: (2024)

Extragradient Method for $(L_0, L_1)$-Lipschitz Root-finding Problems
by: Choudhury, Sayantan, et al.
Published: (2025)

Sharpness-Aware Minimization: General Analysis and Improved Rates
by: Oikonomou, Dimitris, et al.
Published: (2025)

Constant Stepsize Q-learning: Distributional Convergence, Bias and Extrapolation
by: Zhang, Yixuan, et al.
Published: (2024)

Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
by: Agafonov, Artem, et al.
Published: (2025)

Online Stochastic Gradient Methods Under Sub-Weibull Noise and the Polyak-Łojasiewicz Condition
by: Kim, Seunghyun, et al.
Published: (2021)

Acceleration by Stepsize Hedging II: Silver Stepsize Schedule for Smooth Convex Optimization
by: Altschuler, Jason M., et al.
Published: (2023)

Quantized Distributed Nonconvex Optimization Algorithms with Linear Convergence under the Polyak--$Ł$ojasiewicz Condition
by: Xu, Lei, et al.
Published: (2022)

Position: Adopt Constraints Over Fixed Penalties in Deep Learning
by: Ramirez, Juan, et al.
Published: (2025)

Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction
by: Rodomanov, Anton, et al.
Published: (2024)

Achieving Near-Optimal Convergence for Distributed Minimax Optimization with Adaptive Stepsizes
by: Huang, Yan, et al.
Published: (2024)

Bias and Extrapolation in Markovian Linear Stochastic Approximation with Constant Stepsizes
by: Huo, Dongyan, et al.
Published: (2022)