Cover Image

Sparse Polyak: an adaptive step size rule for high-dimensional M-estimation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Qiao, Tianqi, Maros, Marie
Format:	Preprint
Published:	2025
Subjects:	Optimization and Control Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sparse Polyak with optimal thresholding operators for high-dimensional M-estimation
by: Qiao, Tianqi, et al.
Published: (2025)

Decentralized Sparse Linear Regression via Gradient-Tracking: Linear Convergence and Statistical Guarantees
by: Maros, Marie, et al.
Published: (2022)

Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance
by: Oikonomou, Dimitris, et al.
Published: (2024)

Glocal Smoothness: Line search and adaptive step sizes can help in theory too!
by: Fox, Curtis, et al.
Published: (2025)

Adaptive Sharpness-Aware Minimization with a Polyak-type Step size: A Theory-Grounded Scheduler
by: Oikonomou, Dimitris, et al.
Published: (2026)

Constrained Online Convex Optimization with Polyak Feasibility Steps
by: Hutchinson, Spencer, et al.
Published: (2025)

Non-convex Stochastic Composite Optimization with Polyak Momentum
by: Gao, Yuan, et al.
Published: (2024)

Parameter-free Clipped Gradient Descent Meets Polyak
by: Takezawa, Yuki, et al.
Published: (2024)

New Perspectives on the Polyak Stepsize: Surrogate Functions and Negative Results
by: Orabona, Francesco, et al.
Published: (2025)

Faster Stochastic Algorithms for Minimax Optimization under Polyak--Łojasiewicz Conditions
by: Chen, Lesi, et al.
Published: (2023)

Minimisation of Polyak-Łojasewicz Functions Using Random Zeroth-Order Oracles
by: Farzin, Amir Ali, et al.
Published: (2024)

On the Complexity of Finite-Sum Smooth Optimization under the Polyak-Łojasiewicz Condition
by: Bai, Yunyan, et al.
Published: (2024)

Adaptive SGD with Line-Search and Polyak Stepsizes: Nonconvex Convergence and Accelerated Rates
by: Wu, Haotian
Published: (2025)

Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults
by: Phunyaphibarn, Prin, et al.
Published: (2023)

SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms
by: Abdukhakimov, Farshed, et al.
Published: (2023)

High-Probability Bounds for SGD under the Polyak-Lojasiewicz Condition with Markovian Noise
by: Kar, Avik, et al.
Published: (2026)

A Local Polyak-Lojasiewicz and Descent Lemma of Gradient Descent For Overparametrized Linear Models
by: Xu, Ziqing, et al.
Published: (2025)

New logarithmic step size for stochastic gradient descent
by: Shamaee, M. Soheil, et al.
Published: (2024)

Shuffling the Data, Stretching the Step-size: Sharper Bias in constant step-size SGD
by: Emmanouilidis, Konstantinos, et al.
Published: (2026)

Exact Convergence rate of the subgradient method by using Polyak step size
by: Zamani, Moslem, et al.
Published: (2024)

Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients
by: Oikonomou, Dimitris, et al.
Published: (2025)

Convergence and concentration properties of constant step-size SGD through Markov chains
by: Merad, Ibrahim, et al.
Published: (2023)

Convergence rates of stochastic gradient method with independent sequences of step-size and momentum weight
by: Hwang, Wen-Liang
Published: (2024)

A theoretical and empirical study of new adaptive algorithms with additional momentum steps and shifted updates for stochastic non-convex optimization
by: Alecsa, Cristian Daniel
Published: (2021)

Convergence of projected stochastic natural gradient variational inference for various step size and sample or batch size schedules
by: Guilmeau, Thomas, et al.
Published: (2026)

On the Convergence of the Gradient Descent Method with Stochastic Fixed-point Rounding Errors under the Polyak-Lojasiewicz Inequality
by: Xia, Lu, et al.
Published: (2023)

Combining additivity and active subspaces for high-dimensional Gaussian process modeling
by: Binois, Mickael, et al.
Published: (2024)

Sparse-ProxSkip: Accelerated Sparse-to-Sparse Training in Federated Learning
by: Meinhardt, Georg, et al.
Published: (2024)

Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent
by: Vaswani, Sharan, et al.
Published: (2021)

Dissipative Gradient Descent Ascent Method: A Control Theory Inspired Algorithm for Min-max Optimization
by: Zheng, Tianqi, et al.
Published: (2024)

On subdifferential chain rule of matrix factorization and beyond
by: Guan, Jiewen, et al.
Published: (2024)

Model approximation in MDPs with unbounded per-step cost
by: Bozkurt, Berk, et al.
Published: (2024)

Follow The Approximate Sparse Leader for No-Regret Online Sparse Linear Approximation
by: Mukhopadhyay, Samrat, et al.
Published: (2025)

First-Order Sparse Convex Optimization: Better Rates with Sparse Updates
by: Garber, Dan
Published: (2025)

Computing the Bias of Constant-step Stochastic Approximation with Markovian Noise
by: Allmeier, Sebastian, et al.
Published: (2024)

On the SAGA algorithm with decreasing step
by: Fredes, Luis, et al.
Published: (2024)

Off-policy estimation with adaptively collected data: the power of online learning
by: Lee, Jeonghwan, et al.
Published: (2024)

SPP-SBL: Space-Power Prior Sparse Bayesian Learning for Block Sparse Recovery
by: Zhang, Yanhao, et al.
Published: (2025)

Dimension-adapted Momentum Outscales SGD
by: Ferbach, Damien, et al.
Published: (2025)

Bi-Sparse Unsupervised Feature Selection
by: Xiu, Xianchao, et al.
Published: (2024)