Copertina

Iteration and Stochastic First-order Oracle Complexities of Stochastic Gradient Descent using Constant and Decaying Learning Rates

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Imaizumi, Kento, Iiduka, Hideaki
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Adaptive Batch Size and Learning Rate Scheduler for Stochastic Gradient Descent Based on Minimization of Stochastic First-order Oracle Complexity
di: Umeda, Hikaru, et al.
Pubblicazione: (2025)

Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent
di: Umeda, Hikaru, et al.
Pubblicazione: (2024)

Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent
di: Sato, Naoki, et al.
Pubblicazione: (2024)

Both Asymptotic and Non-Asymptotic Convergence of Quasi-Hyperbolic Momentum using Increasing Batch Size
di: Imaizumi, Kento, et al.
Pubblicazione: (2025)

Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum
di: Kamo, Keisuke, et al.
Pubblicazione: (2025)

Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Size
di: Oowada, Kanata, et al.
Pubblicazione: (2025)

Using Stochastic Gradient Descent to Smooth Nonconvex Functions: Analysis of Implicit Graduated Optimization
di: Sato, Naoki, et al.
Pubblicazione: (2023)

Relationship between Batch Size and Number of Steps Needed for Nonconvex Optimization of Stochastic Gradient Descent using Armijo Line Search
di: Tsukada, Yuki, et al.
Pubblicazione: (2023)

Convergence of Sharpness-Aware Minimization Algorithms using Increasing Batch Size and Decaying Learning Rate
di: Harada, Hinata, et al.
Pubblicazione: (2024)

Optimal Growth Schedules for Batch Size and Learning Rate in SGD that Reduce SFO Complexity
di: Umeda, Hikaru, et al.
Pubblicazione: (2025)

Accelerating SGDM via Learning Rate and Batch Size Schedules: A Lyapunov-Based Analysis
di: Kondo, Yuichi, et al.
Pubblicazione: (2025)

Learning a Single Index Model from Anisotropic Data with vanilla Stochastic Gradient Descent
di: Braun, Guillaume, et al.
Pubblicazione: (2025)

Dichotomy of Feature Learning and Unlearning: Fast-Slow Analysis on Neural Networks with Stochastic Gradient Descent
di: Imai, Shota, et al.
Pubblicazione: (2026)

Improved Convergence Rates of Muon Optimizer for Nonconvex Optimization
di: Nagashima, Shuntaro, et al.
Pubblicazione: (2026)

Muon Converges under Heavy-Tailed Noise: Nonconvex Hölder-Smooth Empirical Risk Minimization
di: Iiduka, Hideaki
Pubblicazione: (2026)

Convergence Rate for the Last Iterate of Stochastic Gradient Descent Schemes
di: Hudiani, Marcel
Pubblicazione: (2025)

Scaled Conjugate Gradient Method for Nonconvex Optimization in Deep Neural Networks
di: Sato, Naoki, et al.
Pubblicazione: (2024)

Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization
di: Kassing, Sebastian, et al.
Pubblicazione: (2025)

AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent
di: Surjanovic, Nikola, et al.
Pubblicazione: (2025)

The Sample Complexity of Gradient Descent in Stochastic Convex Optimization
di: Livni, Roi
Pubblicazione: (2024)

Explicit and Implicit Graduated Optimization in Deep Neural Networks
di: Sato, Naoki, et al.
Pubblicazione: (2024)

Convergence Analysis of SGD under Expected Smoothness
di: Kawamoto, Yuta, et al.
Pubblicazione: (2025)

Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approach
di: Sato, Naoki, et al.
Pubblicazione: (2026)

On the Generalization of Stochastic Gradient Descent with Momentum
di: Ramezani-Kebrya, Ali, et al.
Pubblicazione: (2018)

Stochastic Adaptive Gradient Descent Without Descent
di: Aujol, Jean-François, et al.
Pubblicazione: (2025)

Towards Learning Stochastic Population Models by Gradient Descent
di: Kreikemeyer, Justin N., et al.
Pubblicazione: (2024)

Personalized Federated Learning with Exact Stochastic Gradient Descent
di: Nikoloutsopoulos, Sotirios, et al.
Pubblicazione: (2022)

Learning Curves of Stochastic Gradient Descent in Kernel Regression
di: Zhang, Haihan, et al.
Pubblicazione: (2025)

Adjacent Leader Decentralized Stochastic Gradient Descent
di: He, Haoze, et al.
Pubblicazione: (2024)

Stochastic Gradient Descent for Nonparametric Additive Regression
di: Chen, Xin, et al.
Pubblicazione: (2024)

Stochastic Gradient Descent with Adaptive Data
di: Che, Ethan, et al.
Pubblicazione: (2024)

A Bootstrap Perspective on Stochastic Gradient Descent
di: Lan, Hongjian, et al.
Pubblicazione: (2025)

Bolstering Stochastic Gradient Descent with Model Building
di: Birbil, S. Ilker, et al.
Pubblicazione: (2021)

Descend or Rewind? Stochastic Gradient Descent Unlearning
di: Mu, Siqiao, et al.
Pubblicazione: (2025)

Stochastic Gradient Descent with Strategic Querying
di: Jiang, Nanfei, et al.
Pubblicazione: (2025)

Stochastic Differential Equations models for Least-Squares Stochastic Gradient Descent
di: Schertzer, Adrien, et al.
Pubblicazione: (2024)

High-Dimensional Limit of Stochastic Gradient Flow via Dynamical Mean-Field Theory
di: Nishiyama, Sota, et al.
Pubblicazione: (2026)

Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent
di: Lin, Jihao Andreas, et al.
Pubblicazione: (2023)

Mini-Batch Stochastic Halpern Algorithm for Nonexpansive Fixed point Problems
di: Iiduka, Hideaki
Pubblicazione: (2026)

Trustworthiness of Stochastic Gradient Descent in Distributed Learning
di: Li, Hongyang, et al.
Pubblicazione: (2024)