Image de couverture de livre

General framework for online-to-nonconvex conversion: Schedule-free SGD is also effective for nonconvex optimization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Ahn, Kwangjun, Magakyan, Gagik, Cutkosky, Ashok
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Adam with model exponential moving average is effective for nonconvex optimization
par: Ahn, Kwangjun, et autres
Publié: (2024)

Dion: Distributed Orthonormalized Updates
par: Ahn, Kwangjun, et autres
Publié: (2025)

Does SGD really happen in tiny subspaces?
par: Song, Minhak, et autres
Publié: (2024)

Convergence of SGD with momentum in the nonconvex case: A time window-based analysis
par: Qiu, Junwen, et autres
Publié: (2024)

Parameter-free Mirror Descent
par: Jacobsen, Andrew, et autres
Publié: (2022)

The inexact power augmented Lagrangian method for constrained nonconvex optimization
par: Bodard, Alexander, et autres
Publié: (2024)

Newton-CG methods for nonconvex unconstrained optimization with Hölder continuous Hessian
par: He, Chuan, et autres
Publié: (2023)

Block majorization-minimization with diminishing radius for constrained nonsmooth nonconvex optimization
par: Lyu, Hanbaek, et autres
Publié: (2020)

A stochastic smoothing framework for nonconvex-nonconcave min-sum-max problems with applications to Wasserstein distributionally robust optimization
par: Liu, Wei, et autres
Publié: (2025)

Avoiding strict saddle points of nonconvex regularized problems
par: Bai, Luwei, et autres
Publié: (2024)

Projected gradient methods for nonconvex and stochastic smooth optimization: new complexities and auto-conditioned stepsizes
par: Lan, Guanghui, et autres
Publié: (2024)

Random Scaling and Momentum for Non-smooth Non-convex Optimization
par: Zhang, Qinzi, et autres
Publié: (2024)

Fully Unconstrained Online Learning
par: Cutkosky, Ashok, et autres
Publié: (2024)

Unconstrained Robust Online Convex Optimization
par: Zhang, Jiujia, et autres
Publié: (2025)

Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training
par: Song, Minhak, et autres
Publié: (2025)

Nonasymptotic analysis of Stochastic Gradient Hamiltonian Monte Carlo under local conditions for nonconvex optimization
par: Akyildiz, Ömer Deniz, et autres
Publié: (2020)

On exploration of an interior mirror descent flow for stochastic nonconvex constrained problem
par: Ding, Kuangyu, et autres
Publié: (2025)

A randomized algorithm for nonconvex minimization with inexact evaluations and complexity guarantees
par: Li, Shuyao, et autres
Publié: (2023)

Reevaluating Theoretical Analysis Methods for Optimization in Deep Learning
par: Tran, Hoang, et autres
Publié: (2024)

Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion
par: Cutkosky, Ashok, et autres
Publié: (2023)

Simple linesearch-free first-order methods for nonconvex optimization
par: Yagishita, Shotaro, et autres
Publié: (2025)

Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise
par: Ahn, Kwangjun, et autres
Publié: (2024)

The Road Less Scheduled
par: Defazio, Aaron, et autres
Publié: (2024)

An accelerated first-order regularized momentum descent ascent algorithm for stochastic nonconvex-concave minimax problems
par: Zhang, Huiling, et autres
Publié: (2023)

How to escape sharp minima with random perturbations
par: Ahn, Kwangjun, et autres
Publié: (2023)

Certifying optimality in nonconvex robust PCA
par: Gong, Pinxi, et autres
Publié: (2026)

Long-time dynamics and universality of nonconvex gradient descent
par: Han, Qiyang
Publié: (2025)

Scenario approach for minmax optimization with emphasis on the nonconvex case: positive results and caveats
par: K, Mishal Assif P, et autres
Publié: (2019)

Private Zeroth-Order Nonsmooth Nonconvex Optimization
par: Zhang, Qinzi, et autres
Publié: (2024)

Linear attention is (maybe) all you need (to understand transformer optimization)
par: Ahn, Kwangjun, et autres
Publié: (2023)

A single-loop SPIDER-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth optimization
par: Liu, Wei, et autres
Publié: (2025)

Optimal and parameter-free gradient minimization methods for convex and nonconvex optimization
par: Lan, Guanghui, et autres
Publié: (2023)

Adaptive directional decomposition methods for nonconvex constrained optimization
par: Shi, Qiankun, et autres
Publié: (2025)

Stochastic ADMM with batch size adaptation for nonconvex nonsmooth optimization
par: Jin, Jiachen, et autres
Publié: (2025)

An effective subgradient algorithm via Mifflin's line search for nonsmooth nonconvex multiobjective optimization
par: Maleknia, Morteza, et autres
Publié: (2024)

Optimal Growth Schedules for Batch Size and Learning Rate in SGD that Reduce SFO Complexity
par: Umeda, Hikaru, et autres
Publié: (2025)

A deterministic optimization algorithm for nonconvex and combinatorial bi-objective programming
par: Lee, Ye Seol, et autres
Publié: (2024)

Distributed nonconvex optimization for control of water networks with time-coupling constraints
par: Jenks, Bradley, et autres
Publié: (2023)

Nonsmooth nonconvex stochastic heavy ball
par: Le, Tam
Publié: (2023)

Subgradient sampling for nonsmooth nonconvex minimization
par: Bolte, Jérôme, et autres
Publié: (2022)