Copertina

Optimal Projection-Free Adaptive SGD for Matrix Optimization

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Kovalev, Dmitry
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Optimization and Control Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

SGD with Adaptive Preconditioning: Unified Analysis and Momentum Acceleration
di: Kovalev, Dmitry
Pubblicazione: (2025)

Non-Euclidean SGD for Structured Optimization: Unified Analysis and Improved Rates
di: Kovalev, Dmitry, et al.
Pubblicazione: (2025)

Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimization
di: Borodich, Ekaterina, et al.
Pubblicazione: (2025)

Stochastic Non-Smooth Convex Optimization with Unbounded Gradients
di: Kovalev, Dmitry
Pubblicazione: (2026)

On Linear Convergence in Smooth Convex-Concave Bilinearly-Coupled Saddle-Point Optimization: Lower Bounds and Optimal Algorithms
di: Kovalev, Dmitry, et al.
Pubblicazione: (2024)

Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization
di: Kovalev, Dmitry
Pubblicazione: (2025)

Lower Bounds and Optimal Algorithms for Non-Smooth Convex Decentralized Optimization over Time-Varying Networks
di: Kovalev, Dmitry, et al.
Pubblicazione: (2024)

Sign-SGD via Parameter-Free Optimization
di: Medyakov, Daniil, et al.
Pubblicazione: (2025)

The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization
di: Zhang, Haihan, et al.
Pubblicazione: (2024)

SGD with memory: fundamental properties and stochastic acceleration
di: Yarotsky, Dmitry, et al.
Pubblicazione: (2024)

Making SGD Parameter-Free
di: Carmon, Yair, et al.
Pubblicazione: (2022)

Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
di: Tyurin, Alexander, et al.
Pubblicazione: (2024)

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
di: Dahan, Tehila, et al.
Pubblicazione: (2023)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
di: Xie, Shengping, et al.
Pubblicazione: (2025)

Muon is Provably Faster with Momentum Variance Reduction
di: Qian, Xun, et al.
Pubblicazione: (2025)

On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
di: Sahu, Sharan, et al.
Pubblicazione: (2026)

SGD with Partial Hessian for Deep Neural Networks Optimization
di: Sun, Ying, et al.
Pubblicazione: (2024)

Bias-Optimal Bounds for SGD: A Computer-Aided Lyapunov Analysis
di: Cortild, Daniel, et al.
Pubblicazione: (2025)

ROOT-SGD: Sharp Nonasymptotics and Near-Optimal Asymptotics in a Single Algorithm
di: Li, Chris Junchi, et al.
Pubblicazione: (2020)

On the Convergence of DP-SGD with Adaptive Clipping
di: Shulgin, Egor, et al.
Pubblicazione: (2024)

Adaptive SGD with Line-Search and Polyak Stepsizes: Nonconvex Convergence and Accelerated Rates
di: Wu, Haotian
Pubblicazione: (2025)

Optimal Growth Schedules for Batch Size and Learning Rate in SGD that Reduce SFO Complexity
di: Umeda, Hikaru, et al.
Pubblicazione: (2025)

Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Acceleration
di: Khaled, Ahmed, et al.
Pubblicazione: (2025)

SketchySGD: Reliable Stochastic Optimization via Randomized Curvature Estimates
di: Frangella, Zachary, et al.
Pubblicazione: (2022)

Proactive DP: A Multple Target Optimization Framework for DP-SGD
di: van Dijk, Marten, et al.
Pubblicazione: (2021)

A Minibatch-SGD-Based Learning Meta-Policy for Inventory Systems with Myopic Optimal Policy
di: Lyu, Jiameng, et al.
Pubblicazione: (2024)

Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization
di: Armacki, Aleksandar, et al.
Pubblicazione: (2026)

Leveraging Coordinate Momentum in SignSGD and Muon: Memory-Optimized Zero-Order
di: Petrov, Egor, et al.
Pubblicazione: (2025)

Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsize
di: Li, Chris Junchi
Pubblicazione: (2024)

Projection-Free Online Convex Optimization with Time-Varying Constraints
di: Garber, Dan, et al.
Pubblicazione: (2024)

Perturbed Iterate SGD for Lipschitz Continuous Loss Functions with Numerical Error and Adaptive Step Sizes
di: Metel, Michael R.
Pubblicazione: (2022)

On the Trajectories of SGD Without Replacement
di: Beneventano, Pierfrancesco
Pubblicazione: (2023)

$μ^2$-SGD: Stable Stochastic Optimization via a Double Momentum Mechanism
di: Dahan, Tehila, et al.
Pubblicazione: (2023)

Adaptive Matrix Online Learning through Smoothing with Guarantees for Nonsmooth Nonconvex Optimization
di: Jiang, Ruichen, et al.
Pubblicazione: (2026)

Projection-Free Functional Constrained Optimization for Risk Aversion and Sparsity Control
di: Cheng, Yi, et al.
Pubblicazione: (2022)

Optimal Hessian/Jacobian-Free Nonconvex-PL Bilevel Optimization
di: Huang, Feihu
Pubblicazione: (2024)

Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization
di: Jiang, Ruichen, et al.
Pubblicazione: (2024)

Dimension-adapted Momentum Outscales SGD
di: Ferbach, Damien, et al.
Pubblicazione: (2025)

Heavy-Tail Phenomenon in Decentralized SGD
di: Gurbuzbalaban, Mert, et al.
Pubblicazione: (2022)

Demystifying SGD with Doubly Stochastic Gradients
di: Kim, Kyurae, et al.
Pubblicazione: (2024)