Imagen de Portada

Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Choudhury, Sayantan, Tupitsa, Nazarii, Loizou, Nicolas, Horvath, Samuel, Takac, Martin, Gorbunov, Eduard
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence Optimization and Control
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Federated Learning Can Find Friends That Are Advantageous
por: Tupitsa, Nazarii, et al.
Publicado: (2024)

Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity
por: Gorbunov, Eduard, et al.
Publicado: (2024)

Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
por: Chezhegov, Savelii, et al.
Publicado: (2024)

Last Iterate Convergence of AdaGrad-Norm for Convex Non-Smooth Optimization
por: Preobrazhenskaia, Margarita, et al.
Publicado: (2026)

CaCuTe: Casual Cubic-Model Technique for Faster Optimization
por: Tupitsa, Nazarii
Publicado: (2025)

AdaGrad-Diff: A New Version of the Adaptive Gradient Algorithm
por: Bojovic, Matia, et al.
Publicado: (2026)

Extragradient Method for $(L_0, L_1)$-Lipschitz Root-finding Problems
por: Choudhury, Sayantan, et al.
Publicado: (2025)

A Riemannian AdaGrad-Norm Method
por: Bento, Glaydston de C., et al.
Publicado: (2025)

AdaGrad under Anisotropic Smoothness
por: Liu, Yuxing, et al.
Publicado: (2024)

Revisiting Convergence of AdaGrad with Relaxed Assumptions
por: Hong, Yusu, et al.
Publicado: (2024)

AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates
por: Zhang, Minxin, et al.
Publicado: (2025)

Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction
por: Rodomanov, Anton, et al.
Publicado: (2024)

Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations
por: Heredia, Carlos
Publicado: (2024)

Multiplayer Federated Learning: Reaching Equilibrium with Less Communication
por: Yoon, TaeHo, et al.
Publicado: (2025)

Linear Convergence Rate in Convex Setup is Possible! Gradient Descent Method Variants under $(L_0,L_1)$-Smoothness
por: Lobanov, Aleksandr, et al.
Publicado: (2024)

Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates
por: Zhang, Siqi, et al.
Publicado: (2023)

Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization
por: Demidovich, Yury, et al.
Publicado: (2024)

Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
por: Jiang, Ruichen, et al.
Publicado: (2024)

Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad
por: Liu, Zijian
Publicado: (2026)

AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size
por: Ostroukhov, Petr, et al.
Publicado: (2024)

Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters
por: Yukhimchuk, Alexander, et al.
Publicado: (2026)

Next-Generation Iterative Algorithms for Large-Scale Min-Max Optimization: Design and Analysis
por: Choudhury, Sayantan
Publicado: (2025)

Convergence of Clipped-SGD for Convex $(L_0,L_1)$-Smooth Optimization with Heavy-Tailed Noise
por: Chezhegov, Savelii, et al.
Publicado: (2025)

Stochastic Optimization and Data Science
por: Avetisyan, Arutyun, et al.
Publicado: (2026)

Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
por: Choudhury, Sayantan, et al.
Publicado: (2026)

Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy
por: Islamov, Rustem, et al.
Publicado: (2025)

Loss-Transformation Invariance in the Damped Newton Method
por: Shestakov, Alexander, et al.
Publicado: (2025)

Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
por: Khah, Saleh Vatan, et al.
Publicado: (2025)

Polyak Stepsize: Estimating Optimal Functional Values Without Parameters or Prior Knowledge
por: Abdukhakimov, Farshed, et al.
Publicado: (2025)

Exploring Jacobian Inexactness in Second-Order Methods for Variational Inequalities: Lower Bounds, Optimal Algorithms and Quasi-Newton Approximations
por: Agafonov, Artem, et al.
Publicado: (2024)

Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences
por: Malinovsky, Grigory, et al.
Publicado: (2023)

LoFT: Low-Rank Adaptation That Behaves Like Full Fine-Tuning
por: Tastan, Nurbek, et al.
Publicado: (2025)

Stability and convergence analysis of AdaGrad for non-convex optimization via novel stopping time-based techniques
por: Jin, Ruinan, et al.
Publicado: (2024)

On Solving Minimization and Min-Max Problems by First-Order Methods with Relative Error in Gradients
por: Vasin, Artem, et al.
Publicado: (2025)

SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms
por: Abdukhakimov, Farshed, et al.
Publicado: (2023)

Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
por: Agafonov, Artem, et al.
Publicado: (2025)

Recursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimization
por: Gratton, Serge, et al.
Publicado: (2025)

High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise
por: Gorbunov, Eduard, et al.
Publicado: (2023)

Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis
por: Luo, Ruichen, et al.
Publicado: (2025)

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods
por: Lau, Tim Tsz-Kit, et al.
Publicado: (2024)