Image de couverture de livre

AdaGrad under Anisotropic Smoothness

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Liu, Yuxing, Pan, Rui, Zhang, Tong
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Revisiting Convergence of AdaGrad with Relaxed Assumptions
par: Hong, Yusu, et autres
Publié: (2024)

AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates
par: Zhang, Minxin, et autres
Publié: (2025)

AdaGrad-Diff: A New Version of the Adaptive Gradient Algorithm
par: Bojovic, Matia, et autres
Publié: (2026)

Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad
par: Liu, Zijian
Publié: (2026)

Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations
par: Heredia, Carlos
Publié: (2024)

Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
par: Chezhegov, Savelii, et autres
Publié: (2024)

Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise
par: Pan, Rui, et autres
Publié: (2023)

Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad
par: Choudhury, Sayantan, et autres
Publié: (2024)

Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
par: Jiang, Ruichen, et autres
Publié: (2024)

A Riemannian AdaGrad-Norm Method
par: Bento, Glaydston de C., et autres
Publié: (2025)

Last Iterate Convergence of AdaGrad-Norm for Convex Non-Smooth Optimization
par: Preobrazhenskaia, Margarita, et autres
Publié: (2026)

Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction
par: Rodomanov, Anton, et autres
Publié: (2024)

On the Complexity of Finite-Sum Smooth Optimization under the Polyak-Łojasiewicz Condition
par: Bai, Yunyan, et autres
Publié: (2024)

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods
par: Lau, Tim Tsz-Kit, et autres
Publié: (2024)

Theoretical Analysis on how Learning Rate Warmup Accelerates Convergence
par: Liu, Yuxing, et autres
Publié: (2025)

AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size
par: Ostroukhov, Petr, et autres
Publié: (2024)

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
par: Yu, Dingzhi, et autres
Publié: (2026)

Stability and convergence analysis of AdaGrad for non-convex optimization via novel stopping time-based techniques
par: Jin, Ruinan, et autres
Publié: (2024)

ASGO: Adaptive Structured Gradient Optimization
par: An, Kang, et autres
Publié: (2025)

Unbiased Gradient Low-Rank Projection
par: Pan, Rui, et autres
Publié: (2025)

On the Complexity of Decentralized Smooth Nonconvex Finite-Sum Optimization
par: Luo, Luo, et autres
Publié: (2022)

Dynamic Anisotropic Smoothing for Noisy Derivative-Free Optimization
par: Reifenstein, Sam, et autres
Publié: (2024)

Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less
par: Liu, Yuxing, et autres
Publié: (2026)

Provable Adaptivity of Adam under Non-uniform Smoothness
par: Wang, Bohan, et autres
Publié: (2022)

MGDA Converges under Generalized Smoothness, Provably
par: Zhang, Qi, et autres
Publié: (2024)

An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
par: Gong, Xiaochuan, et autres
Publié: (2024)

On the Convergence of Adam-Type Algorithm for Bilevel Optimization under Unbounded Smoothness
par: Gong, Xiaochuan, et autres
Publié: (2025)

GradPower: Powering Gradients for Faster Language Model Pre-Training
par: Wang, Jinbo, et autres
Publié: (2025)

Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis
par: Hao, Jie, et autres
Publié: (2024)

Gradient-Variation Online Learning under Generalized Smoothness
par: Xie, Yan-Feng, et autres
Publié: (2024)

Decentralized Stochastic Nonconvex Optimization under the Relaxed Smoothness
par: Luo, Luo, et autres
Publié: (2025)

On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond
par: Wang, Bohan, et autres
Publié: (2024)

A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
par: Gong, Xiaochuan, et autres
Publié: (2024)

AdaFisher: Adaptive Second Order Optimization via Fisher Information
par: Gomes, Damien Martins, et autres
Publié: (2024)

Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness
par: Crawshaw, Michael, et autres
Publié: (2025)

Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
par: Vaswani, Sharan, et autres
Publié: (2026)

Provably Convergent Decentralized Optimization over Directed Graphs under Generalized Smoothness
par: Bo, Yanan, et autres
Publié: (2026)

PolarGrad: A Class of Matrix-Gradient Optimizers from a Unifying Preconditioning Perspective
par: Lau, Tim Tsz-Kit, et autres
Publié: (2025)

ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting
par: Pan, Rui, et autres
Publié: (2024)

Tight Lower Bounds under Asymmetric High-Order Hölder Smoothness and Uniform Convexity
par: Bai, Cedar Site, et autres
Publié: (2024)