Cover Image

ROOT-SGD: Sharp Nonasymptotics and Near-Optimal Asymptotics in a Single Algorithm

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Chris Junchi, Mou, Wenlong, Wainwright, Martin J., Jordan, Michael I.
Format:	Preprint
Published:	2020
Subjects:	Optimization and Control Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsize
by: Li, Chris Junchi
Published: (2024)

Optimal and instance-dependent guarantees for Markovian linear stochastic approximation
by: Mou, Wenlong, et al.
Published: (2021)

A General Continuous-Time Formulation of Stochastic ADMM and Its Variants
by: Li, Chris Junchi
Published: (2024)

Accelerated Fully First-Order Methods for Bilevel and Minimax Optimization
by: Li, Chris Junchi
Published: (2024)

Does SGD Seek Flatness or Sharpness? An Exactly Solvable Model
by: Xu, Yizhou, et al.
Published: (2026)

Continuous-time reinforcement learning: ellipticity enables model-free value function approximation
by: Mou, Wenlong
Published: (2026)

Optimal Guarantees for Algorithmic Reproducibility and Gradient Complexity in Convex Optimization
by: Zhang, Liang, et al.
Published: (2023)

An Optimistic Algorithm for Online Convex Optimization with Adversarial Constraints
by: Lekeufack, Jordan, et al.
Published: (2024)

SGD at the Edge of Stability: The Stochastic Sharpness Gap
by: Liao, Fangshuo, et al.
Published: (2026)

Statistical guarantees for continuous-time policy evaluation: blessing of ellipticity and new tradeoffs
by: Mou, Wenlong
Published: (2025)

Is RL fine-tuning harder than regression? A PDE learning approach for diffusion models
by: Mou, Wenlong
Published: (2025)

Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis
by: Cayci, Semih, et al.
Published: (2024)

A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
by: Gong, Xiaochuan, et al.
Published: (2024)

Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
by: Tyurin, Alexander, et al.
Published: (2024)

Sharp High-Probability Rates for Nonlinear SGD under Heavy-Tailed Noise via Symmetrization
by: Armacki, Aleksandar, et al.
Published: (2025)

The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization
by: Zhang, Haihan, et al.
Published: (2024)

Optimal Projection-Free Adaptive SGD for Matrix Optimization
by: Kovalev, Dmitry
Published: (2026)

A Near-Optimal Single-Loop Stochastic Algorithm for Convex Finite-Sum Coupled Compositional Optimization
by: Wang, Bokun, et al.
Published: (2023)

Perseus: A Simple and Optimal High-Order Method for Variational Inequalities
by: Lin, Tianyi, et al.
Published: (2022)

Accelerating Single-Pass SGD for Generalized Linear Prediction
by: Chen, Qian, et al.
Published: (2026)

TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization
by: Li, Xiang, et al.
Published: (2022)

Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces
by: Duan, Yaqi, et al.
Published: (2024)

On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
by: Sahu, Sharan, et al.
Published: (2026)

Near-Optimal Algorithms for Group Distributionally Robust Optimization and Beyond
by: Soma, Tasuku, et al.
Published: (2022)

Bias-Optimal Bounds for SGD: A Computer-Aided Lyapunov Analysis
by: Cortild, Daniel, et al.
Published: (2025)

Asymptotically Optimal Regret for Black-Box Predict-then-Optimize
by: Tan, Samuel, et al.
Published: (2024)

A Specialized Semismooth Newton Method for Kernel-Based Optimal Transport
by: Lin, Tianyi, et al.
Published: (2023)

Two-Timescale Gradient Descent Ascent Algorithms for Nonconvex Minimax Optimization
by: Lin, Tianyi, et al.
Published: (2024)

Nonasymptotic Convergence Rates for Plug-and-Play Methods With MMSE Denoisers
by: Pritchard, Henry, et al.
Published: (2025)

On Bellman equations for continuous-time policy evaluation I: discretization and approximation
by: Mou, Wenlong, et al.
Published: (2024)

Exploiting Exogenous Structure for Sample-Efficient Reinforcement Learning
by: Wan, Jia, et al.
Published: (2024)

Optimal Growth Schedules for Batch Size and Learning Rate in SGD that Reduce SFO Complexity
by: Umeda, Hikaru, et al.
Published: (2025)

Nonasymptotic analysis of Stochastic Gradient Hamiltonian Monte Carlo under local conditions for nonconvex optimization
by: Akyildiz, Ömer Deniz, et al.
Published: (2020)

A Minibatch-SGD-Based Learning Meta-Policy for Inventory Systems with Myopic Optimal Policy
by: Lyu, Jiameng, et al.
Published: (2024)

Near-Optimal Primal-Dual Algorithm for Learning Linear Mixture CMDPs with Adversarial Rewards
by: Yu, Kihyun, et al.
Published: (2026)

A Lower Bound and a Near-Optimal Algorithm for Bilevel Empirical Risk Minimization
by: Dagréou, Mathieu, et al.
Published: (2023)

A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation
by: Zhao, Heyang, et al.
Published: (2023)

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
by: Dahan, Tehila, et al.
Published: (2023)

Making SGD Parameter-Free
by: Carmon, Yair, et al.
Published: (2022)

On the Trajectories of SGD Without Replacement
by: Beneventano, Pierfrancesco
Published: (2023)