Copertina

AdaGrad-Diff: A New Version of the Adaptive Gradient Algorithm

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Bojovic, Matia, Salzo, Saverio, Pontil, Massimiliano
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Optimization and Control
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates
di: Grazzi, Riccardo, et al.
Pubblicazione: (2024)

AdaGrad under Anisotropic Smoothness
di: Liu, Yuxing, et al.
Pubblicazione: (2024)

AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates
di: Zhang, Minxin, et al.
Pubblicazione: (2025)

Revisiting Convergence of AdaGrad with Relaxed Assumptions
di: Hong, Yusu, et al.
Pubblicazione: (2024)

Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad
di: Choudhury, Sayantan, et al.
Pubblicazione: (2024)

Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad
di: Liu, Zijian
Pubblicazione: (2026)

Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations
di: Heredia, Carlos
Pubblicazione: (2024)

Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
di: Chezhegov, Savelii, et al.
Pubblicazione: (2024)

High Probability Bounds for Stochastic Subgradient Schemes with Heavy Tailed Noise
di: Parletta, Daniela A., et al.
Pubblicazione: (2022)

Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
di: Jiang, Ruichen, et al.
Pubblicazione: (2024)

Bilevel learning
di: Grazzi, Riccardo, et al.
Pubblicazione: (2026)

A Riemannian AdaGrad-Norm Method
di: Bento, Glaydston de C., et al.
Pubblicazione: (2025)

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods
di: Lau, Tim Tsz-Kit, et al.
Pubblicazione: (2024)

AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size
di: Ostroukhov, Petr, et al.
Pubblicazione: (2024)

Variance reduction techniques for stochastic proximal point algorithms
di: Traoré, Cheik, et al.
Pubblicazione: (2023)

Last Iterate Convergence of AdaGrad-Norm for Convex Non-Smooth Optimization
di: Preobrazhenskaia, Margarita, et al.
Pubblicazione: (2026)

GeoAdaLer: Geometric Insights into Adaptive Stochastic Gradient Descent Algorithms
di: Eleh, Chinedu, et al.
Pubblicazione: (2024)

Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction
di: Rodomanov, Anton, et al.
Pubblicazione: (2024)

Convergence Properties of Stochastic Hypergradients
di: Grazzi, Riccardo, et al.
Pubblicazione: (2020)

Operator World Models for Reinforcement Learning
di: Novelli, Pietro, et al.
Pubblicazione: (2024)

TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization
di: Li, Xiang, et al.
Pubblicazione: (2022)

An Improved Analysis of the Clipped Stochastic subGradient Method under Heavy-Tailed Noise
di: Parletta, Daniela Angela, et al.
Pubblicazione: (2024)

Stability and convergence analysis of AdaGrad for non-convex optimization via novel stopping time-based techniques
di: Jin, Ruinan, et al.
Pubblicazione: (2024)

AdaSwitch: An Adaptive Switching Meta-Algorithm for Learning-Augmented Bounded-Influence Problems
di: Chen, Xi, et al.
Pubblicazione: (2025)

GradPower: Powering Gradients for Faster Language Model Pre-Training
di: Wang, Jinbo, et al.
Pubblicazione: (2025)

PolarGrad: A Class of Matrix-Gradient Optimizers from a Unifying Preconditioning Perspective
di: Lau, Tim Tsz-Kit, et al.
Pubblicazione: (2025)

AdaFisher: Adaptive Second Order Optimization via Fisher Information
di: Gomes, Damien Martins, et al.
Pubblicazione: (2024)

The iterates of FISTA convergence even under inexact computations and stochastic gradients
di: Salzo, Saverio
Pubblicazione: (2025)

Adaptive Moment Estimation Optimization Algorithm Using Projection Gradient for Deep Learning
di: Li, Yongqi, et al.
Pubblicazione: (2025)

Towards a Systems Theory of Algorithms
di: Dörfler, Florian, et al.
Pubblicazione: (2024)

Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness
di: Crawshaw, Michael, et al.
Pubblicazione: (2025)

Adaptive Conditional Gradient Descent
di: Khademi, Abbas, et al.
Pubblicazione: (2025)

Stochastic Gradient Descent with Adaptive Data
di: Che, Ethan, et al.
Pubblicazione: (2024)

ASGO: Adaptive Structured Gradient Optimization
di: An, Kang, et al.
Pubblicazione: (2025)

Enhanced Adaptive Gradient Algorithms for Nonconvex-PL Minimax Optimization
di: Huang, Feihu, et al.
Pubblicazione: (2023)

Shuffling Momentum Gradient Algorithm for Convex Optimization
di: Tran, Trang H., et al.
Pubblicazione: (2024)

A Sketch-and-Project Analysis of Subsampled Natural Gradient Algorithms
di: Goldshlager, Gil, et al.
Pubblicazione: (2025)

On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
di: Zhou, Dongruo, et al.
Pubblicazione: (2018)

Directional Smoothness and Gradient Methods: Convergence and Adaptivity
di: Mishkin, Aaron, et al.
Pubblicazione: (2024)

Stochastic Adaptive Gradient Descent Without Descent
di: Aujol, Jean-François, et al.
Pubblicazione: (2025)