Cover Image

Step-Size Stability in Stochastic Optimization: A Theoretical Perspective

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Schaipp, Fabian, Gower, Robert M., Taylor, Adrien
Format:	Preprint
Published:	2026
Subjects:	Optimization and Control Machine Learning 90C26
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Tracking the Median of Gradients with a Stochastic Proximal Point Method
by: Schaipp, Fabian, et al.
Published: (2024)

A Semismooth Newton Stochastic Proximal Point Algorithm with Variance Reduction
by: Milzarek, Andre, et al.
Published: (2022)

A Normal Map-Based Proximal Stochastic Gradient Method: Convergence and Identification Properties
by: Qiu, Junwen, et al.
Published: (2023)

SPAM: Stochastic Proximal Point Method with Momentum Variance Reduction for Non-convex Cross-Device Federated Learning
by: Karagulyan, Avetik, et al.
Published: (2024)

A New Random Reshuffling Method for Nonsmooth Nonconvex Finite-sum Optimization
by: Qiu, Junwen, et al.
Published: (2023)

Revisiting Frank-Wolfe for Structured Nonconvex Optimization
by: Maskan, Hoomaan, et al.
Published: (2025)

Stochastic First-Order Methods with Non-smooth and Non-Euclidean Proximal Terms for Nonconvex High-Dimensional Stochastic Optimization
by: Xie, Yue, et al.
Published: (2024)

Optimization without Retraction on the Random Generalized Stiefel Manifold
by: Vary, Simon, et al.
Published: (2024)

Shuffling the Stochastic Mirror Descent via Dual Lipschitz Continuity and Kernel Conditioning
by: Qiu, Junwen, et al.
Published: (2026)

An Algebraically Converging Stochastic Gradient Descent Algorithm for Global Optimization
by: Engquist, Björn, et al.
Published: (2022)

Stochastic Gradient Descent Revisited
by: Louzi, Azar
Published: (2024)

Boosted Stochastic Frank-Wolfe for Constrained Nonconvex Optimization
by: Nandhan, Navil, et al.
Published: (2026)

A KL-based Analysis Framework with Applications to Non-Descent Optimization Methods
by: Qiu, Junwen, et al.
Published: (2024)

FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection
by: He, Yutong, et al.
Published: (2026)

Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates
by: Rammal, Ahmad, et al.
Published: (2023)

Global convergence of gradient descent for phase retrieval
by: Fougereux, Théodore, et al.
Published: (2024)

Policy Gradient Algorithms for Robust MDPs with Non-Rectangular Uncertainty Sets
by: Li, Mengmeng, et al.
Published: (2023)

MoSSP: A Momentum-Based Single-Loop Stochastic Penalty Method for Nonconvex Constrained DC-Regularized Optimization
by: Li, Luxuan, et al.
Published: (2026)

A Fully Parameter-Free Second-Order Algorithm for Convex-Concave Minimax Problems
by: Wang, Junlin, et al.
Published: (2024)

A Large Deviations Perspective on Policy Gradient Algorithms
by: Jongeneel, Wouter, et al.
Published: (2023)

Taming Nonconvex Stochastic Mirror Descent with General Bregman Divergence
by: Fatkhullin, Ilyas, et al.
Published: (2024)

Exact Convex Reformulations of Linear Neural Networks via Completely Positive Lifting
by: Prakhya, Karthik, et al.
Published: (2026)

Global Solutions to Non-Convex Functional Constrained Problems with Hidden Convexity
by: Fatkhullin, Ilyas, et al.
Published: (2025)

Two trust region type algorithms for solving nonconvex-strongly concave minimax problems
by: Yao, Tongliang, et al.
Published: (2024)

Gradient Norm Regularization Second-Order Algorithms for Solving Nonconvex-Strongly Concave Minimax Problems
by: Wang, Jun-Lin, et al.
Published: (2024)

A Barrier-Metric First-Order Method for Linearly Constrained Bilevel Optimization
by: Hong, Tenglong, et al.
Published: (2026)

Global Optimization of Gaussian processes
by: Schweidtmann, Artur M., et al.
Published: (2020)

Convergence Conditions for Stochastic Line Search Based Optimization of Over-parametrized Models
by: Lapucci, Matteo, et al.
Published: (2024)

Scenario approach for minmax optimization with emphasis on the nonconvex case: positive results and caveats
by: K, Mishal Assif P, et al.
Published: (2019)

A Generalized Version of Chung's Lemma and its Applications
by: Jiang, Li, et al.
Published: (2024)

Alternating Iteratively Reweighted $\ell_1$ and Subspace Newton Algorithms for Nonconvex Sparse Optimization
by: Wang, Hao, et al.
Published: (2024)

Effectively Leveraging Momentum Terms in Stochastic Line Search Frameworks for Fast Optimization of Finite-Sum Problems
by: Lapucci, Matteo, et al.
Published: (2024)

Stability and convergence analysis of AdaGrad for non-convex optimization via novel stopping time-based techniques
by: Jin, Ruinan, et al.
Published: (2024)

Stochastic Optimization under Hidden Convexity
by: Fatkhullin, Ilyas, et al.
Published: (2023)

DC Semidefinite Programming and Cone Constrained DC Optimization: Theory and Local Search Methods
by: Dolgopolik, M. V.
Published: (2021)

Riemannian Bilevel Optimization
by: Li, Jiaxiang, et al.
Published: (2024)

Revisiting the Geometrically Decaying Step Size: Linear Convergence for Smooth or Non-Smooth Functions
by: Kim, Jihun
Published: (2025)

High Probability Guarantees for Random Reshuffling
by: Yu, Hengxu, et al.
Published: (2023)

Alternating minimization for square root principal component pursuit
by: Deng, Shengxiang, et al.
Published: (2024)

On Tackling High-Dimensional Nonconvex Stochastic Optimization via Stochastic First-Order Methods with Non-smooth Proximal Terms and Variance Reduction
by: Xie, Yue, et al.
Published: (2025)