Copertina

Accelerating SGDM via Learning Rate and Batch Size Schedules: A Lyapunov-Based Analysis

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Kondo, Yuichi, Iiduka, Hideaki
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Optimal Growth Schedules for Batch Size and Learning Rate in SGD that Reduce SFO Complexity
di: Umeda, Hikaru, et al.
Pubblicazione: (2025)

Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent
di: Umeda, Hikaru, et al.
Pubblicazione: (2024)

Adaptive Batch Size and Learning Rate Scheduler for Stochastic Gradient Descent Based on Minimization of Stochastic First-order Oracle Complexity
di: Umeda, Hikaru, et al.
Pubblicazione: (2025)

Convergence of Sharpness-Aware Minimization Algorithms using Increasing Batch Size and Decaying Learning Rate
di: Harada, Hinata, et al.
Pubblicazione: (2024)

Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum
di: Kamo, Keisuke, et al.
Pubblicazione: (2025)

Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Size
di: Oowada, Kanata, et al.
Pubblicazione: (2025)

Convergence Bound and Critical Batch Size of Muon Optimizer
di: Sato, Naoki, et al.
Pubblicazione: (2025)

Both Asymptotic and Non-Asymptotic Convergence of Quasi-Hyperbolic Momentum using Increasing Batch Size
di: Imaizumi, Kento, et al.
Pubblicazione: (2025)

Relationship between Batch Size and Number of Steps Needed for Nonconvex Optimization of Stochastic Gradient Descent using Armijo Line Search
di: Tsukada, Yuki, et al.
Pubblicazione: (2023)

Iteration and Stochastic First-order Oracle Complexities of Stochastic Gradient Descent using Constant and Decaying Learning Rates
di: Imaizumi, Kento, et al.
Pubblicazione: (2024)

Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approach
di: Sato, Naoki, et al.
Pubblicazione: (2026)

Improved Convergence Rates of Muon Optimizer for Nonconvex Optimization
di: Nagashima, Shuntaro, et al.
Pubblicazione: (2026)

Muon Converges under Heavy-Tailed Noise: Nonconvex Hölder-Smooth Empirical Risk Minimization
di: Iiduka, Hideaki
Pubblicazione: (2026)

Convergence Analysis of SGD under Expected Smoothness
di: Kawamoto, Yuta, et al.
Pubblicazione: (2025)

Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling
di: Meterez, Alexandru, et al.
Pubblicazione: (2025)

Using Stochastic Gradient Descent to Smooth Nonconvex Functions: Analysis of Implicit Graduated Optimization
di: Sato, Naoki, et al.
Pubblicazione: (2023)

Explicit and Implicit Graduated Optimization in Deep Neural Networks
di: Sato, Naoki, et al.
Pubblicazione: (2024)

Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent
di: Sato, Naoki, et al.
Pubblicazione: (2024)

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler
di: Shen, Yikang, et al.
Pubblicazione: (2024)

Scaled Conjugate Gradient Method for Nonconvex Optimization in Deep Neural Networks
di: Sato, Naoki, et al.
Pubblicazione: (2024)

Mini-Batch Stochastic Halpern Algorithm for Nonexpansive Fixed point Problems
di: Iiduka, Hideaki
Pubblicazione: (2026)

Smaller Batches, Bigger Gains? Investigating the Impact of Batch Sizes on Reinforcement Learning Based Real-World Production Scheduling
di: Müller, Arthur, et al.
Pubblicazione: (2024)

Mini-Batch Stochastic Krasnosel'ski\uı-Mann Algorithm for Nonexpansive Fixed Point Problems
di: Iiduka, Hideaki
Pubblicazione: (2026)

Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling
di: Li, Shuaipeng, et al.
Pubblicazione: (2024)

Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit
di: Filatov, Oleg, et al.
Pubblicazione: (2024)

Fast Catch-Up, Late Switching: Optimal Batch Size Scheduling via Functional Scaling Laws
di: Wang, Jinbo, et al.
Pubblicazione: (2026)

One Size Does Not Fit All: Architecture-Aware Adaptive Batch Scheduling with DEBA
di: Belias, François, et al.
Pubblicazione: (2025)

Full-Graph vs. Mini-Batch Training: Comprehensive Analysis from a Batch Size and Fan-Out Size Perspective
di: Liu, Mengfan, et al.
Pubblicazione: (2026)

On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond
di: Wang, Bohan, et al.
Pubblicazione: (2024)

Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism
di: Lau, Tim Tsz-Kit, et al.
Pubblicazione: (2024)

DIVEBATCH: Accelerating Model Training Through Gradient-Diversity Aware Batch Size Adaptation
di: Chen, Yuen, et al.
Pubblicazione: (2025)

AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size
di: Ostroukhov, Petr, et al.
Pubblicazione: (2024)

Collaborative Batch Size Optimization for Federated Learning
di: Geimer, Arno, et al.
Pubblicazione: (2025)

Decoupled Relative Learning Rate Schedules
di: Ludziejewski, Jan, et al.
Pubblicazione: (2025)

Actionable Interpretability via Causal Hypergraphs: Unravelling Batch Size Effects in Deep Learning
di: Sun, Zhongtian, et al.
Pubblicazione: (2025)

Critical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Training
di: Merrill, William, et al.
Pubblicazione: (2025)

Diversified Batch Selection for Training Acceleration
di: Hong, Feng, et al.
Pubblicazione: (2024)

The N-Grammys: Accelerating Autoregressive Inference with Learning-Free Batched Speculation
di: Stewart, Lawrence, et al.
Pubblicazione: (2024)

A Concise Lyapunov Analysis of Nesterov's Accelerated Gradient Method
di: Liu, Jun
Pubblicazione: (2025)

An Adaptive Volatility-based Learning Rate Scheduler
di: Ren, Kieran Chai Kai
Pubblicazione: (2025)