Copertina

Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Rodomanov, Anton, Jiang, Xiaowen, Stich, Sebastian
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Optimization and Control
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates
di: Zhang, Minxin, et al.
Pubblicazione: (2025)

Federated Optimization with Doubly Regularized Drift Correction
di: Jiang, Xiaowen, et al.
Pubblicazione: (2024)

Stabilized Proximal-Point Methods for Federated Optimization
di: Jiang, Xiaowen, et al.
Pubblicazione: (2024)

Non-Convex Federated Optimization under Cost-Aware Client Selection
di: Jiang, Xiaowen, et al.
Pubblicazione: (2025)

Exploiting Similarity for Computation and Communication-Efficient Decentralized Optimization
di: Takezawa, Yuki, et al.
Pubblicazione: (2025)

Non-convex Stochastic Composite Optimization with Polyak Momentum
di: Gao, Yuan, et al.
Pubblicazione: (2024)

A Riemannian AdaGrad-Norm Method
di: Bento, Glaydston de C., et al.
Pubblicazione: (2025)

AdaGrad under Anisotropic Smoothness
di: Liu, Yuxing, et al.
Pubblicazione: (2024)

Revisiting Convergence of AdaGrad with Relaxed Assumptions
di: Hong, Yusu, et al.
Pubblicazione: (2024)

Accelerated Distributed Optimization with Compression and Error Feedback
di: Gao, Yuan, et al.
Pubblicazione: (2025)

Last Iterate Convergence of AdaGrad-Norm for Convex Non-Smooth Optimization
di: Preobrazhenskaia, Margarita, et al.
Pubblicazione: (2026)

Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
di: Jiang, Ruichen, et al.
Pubblicazione: (2024)

Composite Optimization with Error Feedback: the Dual Averaging Approach
di: Gao, Yuan, et al.
Pubblicazione: (2025)

AdaGrad-Diff: A New Version of the Adaptive Gradient Algorithm
di: Bojovic, Matia, et al.
Pubblicazione: (2026)

Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations
di: Heredia, Carlos
Pubblicazione: (2024)

Optimizing $(L_0, L_1)$-Smooth Functions by Gradient Methods
di: Vankov, Daniil, et al.
Pubblicazione: (2024)

DADA: Dual Averaging with Distance Adaptation
di: Moshtaghifar, Mohammad, et al.
Pubblicazione: (2025)

Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
di: Chezhegov, Savelii, et al.
Pubblicazione: (2024)

Universal Gradient Methods for Stochastic Convex Optimization
di: Rodomanov, Anton, et al.
Pubblicazione: (2024)

Efficient Gradient Methods for Distributed Saddle Problems
di: Luo, Ruichen, et al.
Pubblicazione: (2026)

Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad
di: Choudhury, Sayantan, et al.
Pubblicazione: (2024)

Global Complexity Analysis of BFGS
di: Rodomanov, Anton
Pubblicazione: (2024)

Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad
di: Liu, Zijian
Pubblicazione: (2026)

A Universal Transfer Theorem for Convex Optimization Algorithms Using Inexact First-order Oracles
di: Kerger, Phillip, et al.
Pubblicazione: (2024)

Acceleration by Stepsize Hedging II: Silver Stepsize Schedule for Smooth Convex Optimization
di: Altschuler, Jason M., et al.
Pubblicazione: (2023)

FedMuon: Federated Learning with Bias-corrected LMO-based Optimization
di: Takezawa, Yuki, et al.
Pubblicazione: (2025)

Nonasymptotic Analysis of Accelerated Methods With Inexact Oracle Under Absolute Error Bound
di: Liu, Yin, et al.
Pubblicazione: (2024)

Generalization of Silver Stepsize Schedule to Stochastic Optimization
di: Bai, Luwei, et al.
Pubblicazione: (2025)

Accelerated Gradient Descent by Concatenation of Stepsize Schedules
di: Zhang, Zehao, et al.
Pubblicazione: (2024)

Inexact and Implementable Accelerated Newton Proximal Extragradient Method for Convex Optimization
di: Huang, Ziyu, et al.
Pubblicazione: (2024)

Trust-Region Stochastic Optimization with Variance Reduction Technique
di: Zheng, Xinshou
Pubblicazione: (2024)

Adaptive Variance Reduction for Stochastic Optimization under Weaker Assumptions
di: Jiang, Wei, et al.
Pubblicazione: (2024)

Stochastic Approximation with Block Coordinate Optimal Stepsizes
di: Jiang, Tao, et al.
Pubblicazione: (2025)

Abs-Smooth Frank-Wolfe Method: Primal-Dual Analysis, Heavy Ball Momentum, and Inexact Oracles
di: Tadinada, Sri Harshitha, et al.
Pubblicazione: (2026)

Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction
di: Jiang, Wei, et al.
Pubblicazione: (2024)

Variance Reduced Distributed Non-Convex Optimization Using Matrix Stepsizes
di: Li, Hanmin, et al.
Pubblicazione: (2023)

Towards Faster Decentralized Stochastic Optimization with Communication Compression
di: Islamov, Rustem, et al.
Pubblicazione: (2024)

On the Oracle Complexity of a Riemannian Inexact Augmented Lagrangian Method for Riemannian Nonsmooth Composite Problems
di: Xu, Meng, et al.
Pubblicazione: (2024)

Accelerating Level-Value Adjustment for the Polyak Stepsize
di: Liu, Anbang, et al.
Pubblicazione: (2023)

Stability and convergence analysis of AdaGrad for non-convex optimization via novel stopping time-based techniques
di: Jin, Ruinan, et al.
Pubblicazione: (2024)