ROOT-SGD: Sharp Nonasymptotics and Near-Optimal Asymptotics in a Single Algorithm
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | Li, Chris Junchi, Mou, Wenlong, Wainwright, Martin J., Jordan, Michael I. |
|---|---|
| Format: | Preprint |
| Publié: |
2020
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
Documents similaires
Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsize
par: Li, Chris Junchi
Publié: (2024)
par: Li, Chris Junchi
Publié: (2024)
Optimal and instance-dependent guarantees for Markovian linear stochastic approximation
par: Mou, Wenlong, et autres
Publié: (2021)
par: Mou, Wenlong, et autres
Publié: (2021)
A General Continuous-Time Formulation of Stochastic ADMM and Its Variants
par: Li, Chris Junchi
Publié: (2024)
par: Li, Chris Junchi
Publié: (2024)
Accelerated Fully First-Order Methods for Bilevel and Minimax Optimization
par: Li, Chris Junchi
Publié: (2024)
par: Li, Chris Junchi
Publié: (2024)
Does SGD Seek Flatness or Sharpness? An Exactly Solvable Model
par: Xu, Yizhou, et autres
Publié: (2026)
par: Xu, Yizhou, et autres
Publié: (2026)
Continuous-time reinforcement learning: ellipticity enables model-free value function approximation
par: Mou, Wenlong
Publié: (2026)
par: Mou, Wenlong
Publié: (2026)
Optimal Guarantees for Algorithmic Reproducibility and Gradient Complexity in Convex Optimization
par: Zhang, Liang, et autres
Publié: (2023)
par: Zhang, Liang, et autres
Publié: (2023)
An Optimistic Algorithm for Online Convex Optimization with Adversarial Constraints
par: Lekeufack, Jordan, et autres
Publié: (2024)
par: Lekeufack, Jordan, et autres
Publié: (2024)
SGD at the Edge of Stability: The Stochastic Sharpness Gap
par: Liao, Fangshuo, et autres
Publié: (2026)
par: Liao, Fangshuo, et autres
Publié: (2026)
Statistical guarantees for continuous-time policy evaluation: blessing of ellipticity and new tradeoffs
par: Mou, Wenlong
Publié: (2025)
par: Mou, Wenlong
Publié: (2025)
Is RL fine-tuning harder than regression? A PDE learning approach for diffusion models
par: Mou, Wenlong
Publié: (2025)
par: Mou, Wenlong
Publié: (2025)
Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis
par: Cayci, Semih, et autres
Publié: (2024)
par: Cayci, Semih, et autres
Publié: (2024)
A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
par: Gong, Xiaochuan, et autres
Publié: (2024)
par: Gong, Xiaochuan, et autres
Publié: (2024)
Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
par: Tyurin, Alexander, et autres
Publié: (2024)
par: Tyurin, Alexander, et autres
Publié: (2024)
Sharp High-Probability Rates for Nonlinear SGD under Heavy-Tailed Noise via Symmetrization
par: Armacki, Aleksandar, et autres
Publié: (2025)
par: Armacki, Aleksandar, et autres
Publié: (2025)
The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization
par: Zhang, Haihan, et autres
Publié: (2024)
par: Zhang, Haihan, et autres
Publié: (2024)
Optimal Projection-Free Adaptive SGD for Matrix Optimization
par: Kovalev, Dmitry
Publié: (2026)
par: Kovalev, Dmitry
Publié: (2026)
A Near-Optimal Single-Loop Stochastic Algorithm for Convex Finite-Sum Coupled Compositional Optimization
par: Wang, Bokun, et autres
Publié: (2023)
par: Wang, Bokun, et autres
Publié: (2023)
Perseus: A Simple and Optimal High-Order Method for Variational Inequalities
par: Lin, Tianyi, et autres
Publié: (2022)
par: Lin, Tianyi, et autres
Publié: (2022)
Accelerating Single-Pass SGD for Generalized Linear Prediction
par: Chen, Qian, et autres
Publié: (2026)
par: Chen, Qian, et autres
Publié: (2026)
TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization
par: Li, Xiang, et autres
Publié: (2022)
par: Li, Xiang, et autres
Publié: (2022)
Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces
par: Duan, Yaqi, et autres
Publié: (2024)
par: Duan, Yaqi, et autres
Publié: (2024)
On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
par: Sahu, Sharan, et autres
Publié: (2026)
par: Sahu, Sharan, et autres
Publié: (2026)
Near-Optimal Algorithms for Group Distributionally Robust Optimization and Beyond
par: Soma, Tasuku, et autres
Publié: (2022)
par: Soma, Tasuku, et autres
Publié: (2022)
Bias-Optimal Bounds for SGD: A Computer-Aided Lyapunov Analysis
par: Cortild, Daniel, et autres
Publié: (2025)
par: Cortild, Daniel, et autres
Publié: (2025)
Asymptotically Optimal Regret for Black-Box Predict-then-Optimize
par: Tan, Samuel, et autres
Publié: (2024)
par: Tan, Samuel, et autres
Publié: (2024)
A Specialized Semismooth Newton Method for Kernel-Based Optimal Transport
par: Lin, Tianyi, et autres
Publié: (2023)
par: Lin, Tianyi, et autres
Publié: (2023)
Two-Timescale Gradient Descent Ascent Algorithms for Nonconvex Minimax Optimization
par: Lin, Tianyi, et autres
Publié: (2024)
par: Lin, Tianyi, et autres
Publié: (2024)
Nonasymptotic Convergence Rates for Plug-and-Play Methods With MMSE Denoisers
par: Pritchard, Henry, et autres
Publié: (2025)
par: Pritchard, Henry, et autres
Publié: (2025)
On Bellman equations for continuous-time policy evaluation I: discretization and approximation
par: Mou, Wenlong, et autres
Publié: (2024)
par: Mou, Wenlong, et autres
Publié: (2024)
Exploiting Exogenous Structure for Sample-Efficient Reinforcement Learning
par: Wan, Jia, et autres
Publié: (2024)
par: Wan, Jia, et autres
Publié: (2024)
Optimal Growth Schedules for Batch Size and Learning Rate in SGD that Reduce SFO Complexity
par: Umeda, Hikaru, et autres
Publié: (2025)
par: Umeda, Hikaru, et autres
Publié: (2025)
Nonasymptotic analysis of Stochastic Gradient Hamiltonian Monte Carlo under local conditions for nonconvex optimization
par: Akyildiz, Ömer Deniz, et autres
Publié: (2020)
par: Akyildiz, Ömer Deniz, et autres
Publié: (2020)
A Minibatch-SGD-Based Learning Meta-Policy for Inventory Systems with Myopic Optimal Policy
par: Lyu, Jiameng, et autres
Publié: (2024)
par: Lyu, Jiameng, et autres
Publié: (2024)
Near-Optimal Primal-Dual Algorithm for Learning Linear Mixture CMDPs with Adversarial Rewards
par: Yu, Kihyun, et autres
Publié: (2026)
par: Yu, Kihyun, et autres
Publié: (2026)
A Lower Bound and a Near-Optimal Algorithm for Bilevel Empirical Risk Minimization
par: Dagréou, Mathieu, et autres
Publié: (2023)
par: Dagréou, Mathieu, et autres
Publié: (2023)
A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation
par: Zhao, Heyang, et autres
Publié: (2023)
par: Zhao, Heyang, et autres
Publié: (2023)
SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
par: Dahan, Tehila, et autres
Publié: (2023)
par: Dahan, Tehila, et autres
Publié: (2023)
Making SGD Parameter-Free
par: Carmon, Yair, et autres
Publié: (2022)
par: Carmon, Yair, et autres
Publié: (2022)
On the Trajectories of SGD Without Replacement
par: Beneventano, Pierfrancesco
Publié: (2023)
par: Beneventano, Pierfrancesco
Publié: (2023)
Documents similaires
-
Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsize
par: Li, Chris Junchi
Publié: (2024) -
Optimal and instance-dependent guarantees for Markovian linear stochastic approximation
par: Mou, Wenlong, et autres
Publié: (2021) -
A General Continuous-Time Formulation of Stochastic ADMM and Its Variants
par: Li, Chris Junchi
Publié: (2024) -
Accelerated Fully First-Order Methods for Bilevel and Minimax Optimization
par: Li, Chris Junchi
Publié: (2024) -
Does SGD Seek Flatness or Sharpness? An Exactly Solvable Model
par: Xu, Yizhou, et autres
Publié: (2026)