Cover Image

Iteration and Stochastic First-order Oracle Complexities of Stochastic Gradient Descent using Constant and Decaying Learning Rates

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Imaizumi, Kento, Iiduka, Hideaki
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Adaptive Batch Size and Learning Rate Scheduler for Stochastic Gradient Descent Based on Minimization of Stochastic First-order Oracle Complexity
by: Umeda, Hikaru, et al.
Published: (2025)

Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent
by: Umeda, Hikaru, et al.
Published: (2024)

Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent
by: Sato, Naoki, et al.
Published: (2024)

Both Asymptotic and Non-Asymptotic Convergence of Quasi-Hyperbolic Momentum using Increasing Batch Size
by: Imaizumi, Kento, et al.
Published: (2025)

Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum
by: Kamo, Keisuke, et al.
Published: (2025)

Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Size
by: Oowada, Kanata, et al.
Published: (2025)

Using Stochastic Gradient Descent to Smooth Nonconvex Functions: Analysis of Implicit Graduated Optimization
by: Sato, Naoki, et al.
Published: (2023)

Relationship between Batch Size and Number of Steps Needed for Nonconvex Optimization of Stochastic Gradient Descent using Armijo Line Search
by: Tsukada, Yuki, et al.
Published: (2023)

Convergence of Sharpness-Aware Minimization Algorithms using Increasing Batch Size and Decaying Learning Rate
by: Harada, Hinata, et al.
Published: (2024)

Optimal Growth Schedules for Batch Size and Learning Rate in SGD that Reduce SFO Complexity
by: Umeda, Hikaru, et al.
Published: (2025)

Accelerating SGDM via Learning Rate and Batch Size Schedules: A Lyapunov-Based Analysis
by: Kondo, Yuichi, et al.
Published: (2025)

Learning a Single Index Model from Anisotropic Data with vanilla Stochastic Gradient Descent
by: Braun, Guillaume, et al.
Published: (2025)

Dichotomy of Feature Learning and Unlearning: Fast-Slow Analysis on Neural Networks with Stochastic Gradient Descent
by: Imai, Shota, et al.
Published: (2026)

Improved Convergence Rates of Muon Optimizer for Nonconvex Optimization
by: Nagashima, Shuntaro, et al.
Published: (2026)

Muon Converges under Heavy-Tailed Noise: Nonconvex Hölder-Smooth Empirical Risk Minimization
by: Iiduka, Hideaki
Published: (2026)

Convergence Rate for the Last Iterate of Stochastic Gradient Descent Schemes
by: Hudiani, Marcel
Published: (2025)

Scaled Conjugate Gradient Method for Nonconvex Optimization in Deep Neural Networks
by: Sato, Naoki, et al.
Published: (2024)

Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization
by: Kassing, Sebastian, et al.
Published: (2025)

AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent
by: Surjanovic, Nikola, et al.
Published: (2025)

The Sample Complexity of Gradient Descent in Stochastic Convex Optimization
by: Livni, Roi
Published: (2024)

Explicit and Implicit Graduated Optimization in Deep Neural Networks
by: Sato, Naoki, et al.
Published: (2024)

Convergence Analysis of SGD under Expected Smoothness
by: Kawamoto, Yuta, et al.
Published: (2025)

Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approach
by: Sato, Naoki, et al.
Published: (2026)

On the Generalization of Stochastic Gradient Descent with Momentum
by: Ramezani-Kebrya, Ali, et al.
Published: (2018)

Stochastic Adaptive Gradient Descent Without Descent
by: Aujol, Jean-François, et al.
Published: (2025)

Towards Learning Stochastic Population Models by Gradient Descent
by: Kreikemeyer, Justin N., et al.
Published: (2024)

Personalized Federated Learning with Exact Stochastic Gradient Descent
by: Nikoloutsopoulos, Sotirios, et al.
Published: (2022)

Learning Curves of Stochastic Gradient Descent in Kernel Regression
by: Zhang, Haihan, et al.
Published: (2025)

Adjacent Leader Decentralized Stochastic Gradient Descent
by: He, Haoze, et al.
Published: (2024)

Stochastic Gradient Descent for Nonparametric Additive Regression
by: Chen, Xin, et al.
Published: (2024)

Stochastic Gradient Descent with Adaptive Data
by: Che, Ethan, et al.
Published: (2024)

A Bootstrap Perspective on Stochastic Gradient Descent
by: Lan, Hongjian, et al.
Published: (2025)

Bolstering Stochastic Gradient Descent with Model Building
by: Birbil, S. Ilker, et al.
Published: (2021)

Descend or Rewind? Stochastic Gradient Descent Unlearning
by: Mu, Siqiao, et al.
Published: (2025)

Stochastic Gradient Descent with Strategic Querying
by: Jiang, Nanfei, et al.
Published: (2025)

Stochastic Differential Equations models for Least-Squares Stochastic Gradient Descent
by: Schertzer, Adrien, et al.
Published: (2024)

High-Dimensional Limit of Stochastic Gradient Flow via Dynamical Mean-Field Theory
by: Nishiyama, Sota, et al.
Published: (2026)

Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent
by: Lin, Jihao Andreas, et al.
Published: (2023)

Mini-Batch Stochastic Halpern Algorithm for Nonexpansive Fixed point Problems
by: Iiduka, Hideaki
Published: (2026)

Trustworthiness of Stochastic Gradient Descent in Distributed Learning
by: Li, Hongyang, et al.
Published: (2024)