Cover Image

Adaptive Gradient Normalization and Independent Sampling for (Stochastic) Generalized-Smooth Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Yufeng, Tripp, Erin, Sun, Yifan, Zou, Shaofeng, Zhou, Yi
Format:	Preprint
Published:	2024
Subjects:	Optimization and Control Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
by: Zhang, Qi, et al.
Published: (2024)

Nested Stochastic Algorithm for Generalized Sinkhorn distance-Regularized Distributionally Robust Optimization
by: Yang, Yufeng, et al.
Published: (2025)

Gradient-Normalized Smoothness for Optimization with Approximate Hessians
by: Semenov, Andrei, et al.
Published: (2025)

MGDA Converges under Generalized Smoothness, Provably
by: Zhang, Qi, et al.
Published: (2024)

Stochastic Non-Smooth Convex Optimization with Unbounded Gradients
by: Kovalev, Dmitry
Published: (2026)

Faster Gradient Methods for Highly-Smooth Stochastic Bilevel Optimization
by: Chen, Lesi, et al.
Published: (2025)

Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness
by: Crawshaw, Michael, et al.
Published: (2025)

Adaptive Optimization via Momentum on Variance-Normalized Gradients
by: Patitucci, Francisco, et al.
Published: (2026)

Gradient-Variation Online Adaptivity for Accelerated Optimization with Hölder Smoothness
by: Zhao, Yuheng, et al.
Published: (2025)

A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization
by: Zhu, Yuchen, et al.
Published: (2024)

The Sample Complexity of Gradient Descent in Stochastic Convex Optimization
by: Livni, Roi
Published: (2024)

Using Stochastic Gradient Descent to Smooth Nonconvex Functions: Analysis of Implicit Graduated Optimization
by: Sato, Naoki, et al.
Published: (2023)

Directional Smoothness and Gradient Methods: Convergence and Adaptivity
by: Mishkin, Aaron, et al.
Published: (2024)

Stochastic Smoothed Gradient Descent Ascent for Federated Minimax Optimization
by: Shen, Wei, et al.
Published: (2023)

Gradient-Variation Online Learning under Generalized Smoothness
by: Xie, Yan-Feng, et al.
Published: (2024)

Multi-level Monte-Carlo Gradient Methods for Stochastic Optimization with Biased Oracles
by: Hu, Yifan, et al.
Published: (2024)

On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
by: Zhou, Dongruo, et al.
Published: (2018)

A Stochastic-Gradient-based Interior-Point Algorithm for Solving Smooth Bound-Constrained Optimization Problems
by: Curtis, Frank E., et al.
Published: (2023)

Stochastic Gradient Descent with Adaptive Data
by: Che, Ethan, et al.
Published: (2024)

A Stochastic Quasi-Newton Method for Non-convex Optimization with Non-uniform Smoothness
by: Sun, Zhenyu, et al.
Published: (2024)

ASGO: Adaptive Structured Gradient Optimization
by: An, Kang, et al.
Published: (2025)

Decentralized Stochastic Nonconvex Optimization under the Relaxed Smoothness
by: Luo, Luo, et al.
Published: (2025)

A Retrospective Approximation Approach for Smooth Stochastic Optimization
by: Newton, David, et al.
Published: (2021)

Stochastic Adaptive Gradient Descent Without Descent
by: Aujol, Jean-François, et al.
Published: (2025)

Reusing Historical Trajectories in Natural Policy Gradient via Importance Sampling: Convergence and Convergence Rate
by: Lin, Yifan, et al.
Published: (2024)

An Accelerated Gradient Method for Convex Smooth Simple Bilevel Optimization
by: Cao, Jincheng, et al.
Published: (2024)

Distributionally Robust Multi-Objective Optimization
by: Yang, Yufeng, et al.
Published: (2026)

An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
by: Gong, Xiaochuan, et al.
Published: (2024)

Adaptive Step Sizes for Preconditioned Stochastic Gradient Descent
by: Köhne, Frederik, et al.
Published: (2023)

On the Stochastic (Variance-Reduced) Proximal Gradient Method for Regularized Expected Reward Optimization
by: Liang, Ling, et al.
Published: (2024)

Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
by: Vaswani, Sharan, et al.
Published: (2026)

Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping
by: Liu, Zijian, et al.
Published: (2024)

Demystifying SGD with Doubly Stochastic Gradients
by: Kim, Kyurae, et al.
Published: (2024)

Rapid Overfitting of Multi-Pass Stochastic Gradient Descent in Stochastic Convex Optimization
by: Vansover-Hager, Shira, et al.
Published: (2025)

Riemannian Stochastic Gradient Method for Nested Composition Optimization
by: Zhang, Dewei, et al.
Published: (2022)

A Homogenization Approach for Gradient-Dominated Stochastic Optimization
by: Tan, Jiyuan, et al.
Published: (2023)

An Energy-Based Self-Adaptive Learning Rate for Stochastic Gradient Descent: Enhancing Unconstrained Optimization with VAV method
by: Zhang, Jiahao, et al.
Published: (2024)

The Price of Adaptivity in Stochastic Convex Optimization
by: Carmon, Yair, et al.
Published: (2024)

Stochastic Smoothed Primal-Dual Algorithms for Nonconvex Optimization with Linear Inequality Constraints
by: Huang, Ruichuan, et al.
Published: (2025)

Adaptive Variance Reduction for Stochastic Optimization under Weaker Assumptions
by: Jiang, Wei, et al.
Published: (2024)