Copertina

Stochastic Normalized Gradient Descent with Momentum for Large-Batch Training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhao, Shen-Yi, Shi, Chang-Wei, Xie, Yin-Peng, Li, Wu-Jun
Natura:	Preprint
Pubblicazione:	2020
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum
di: Kamo, Keisuke, et al.
Pubblicazione: (2025)

Global Momentum Compression for Sparse Communication in Distributed Learning
di: Shi, Chang-Wei, et al.
Pubblicazione: (2019)

On the Generalization of Stochastic Gradient Descent with Momentum
di: Ramezani-Kebrya, Ali, et al.
Pubblicazione: (2018)

Weighted Averaged Stochastic Gradient Descent: Asymptotic Normality and Optimality
di: Wei, Ziyang, et al.
Pubblicazione: (2023)

Stochastic Gradient Descent with Momentum is Algorithmically Stable
di: Lei, Yunwen, et al.
Pubblicazione: (2026)

Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent
di: Sato, Naoki, et al.
Pubblicazione: (2024)

Adaptive Batch Sizes Using Non-Euclidean Gradient Noise Scales for Stochastic Sign and Spectral Descent
di: Naganuma, Hiroki, et al.
Pubblicazione: (2026)

Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Size
di: Oowada, Kanata, et al.
Pubblicazione: (2025)

Beyond the Mean: Fisher-Orthogonal Projection for Natural Gradient Descent in Large Batch Training
di: Lu, Yishun, et al.
Pubblicazione: (2025)

Central Limit Theorems for Stochastic Gradient Descent Quantile Estimators
di: Wei, Ziyang, et al.
Pubblicazione: (2025)

Statistical Guarantees for High-Dimensional Stochastic Gradient Descent
di: Li, Jiaqi, et al.
Pubblicazione: (2025)

Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent
di: Umeda, Hikaru, et al.
Pubblicazione: (2024)

Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent
di: Deng, Xiaoge, et al.
Pubblicazione: (2023)

Grams: Gradient Descent with Adaptive Momentum Scaling
di: Cao, Yang, et al.
Pubblicazione: (2024)

First and Second Order Approximations to Stochastic Gradient Descent Methods with Momentum Terms
di: Lu, Eric
Pubblicazione: (2025)

Training-Time Batch Normalization Reshapes Local Partition Geometry in Piecewise-Affine Networks
di: Qi, Xuan, et al.
Pubblicazione: (2026)

Ordered Momentum for Asynchronous SGD
di: Shi, Chang-Wei, et al.
Pubblicazione: (2024)

Algorithmic Stability of Stochastic Gradient Descent with Momentum under Heavy-Tailed Noise
di: Dang, Thanh, et al.
Pubblicazione: (2025)

Stochastic Smoothed Gradient Descent Ascent for Federated Minimax Optimization
di: Shen, Wei, et al.
Pubblicazione: (2023)

Stochastic Gradient Descent for Two-layer Neural Networks
di: Cao, Dinghao, et al.
Pubblicazione: (2024)

Ordered Local Momentum for Asynchronous Distributed Learning under Arbitrary Delays
di: Shi, Chang-Wei, et al.
Pubblicazione: (2026)

Coupling-based Convergence Diagnostic and Stepsize Scheme for Stochastic Gradient Descent
di: Li, Xiang, et al.
Pubblicazione: (2024)

Full-Batch Gradient Descent Outperforms One-Pass SGD: Sample Complexity Separation in Single-Index Learning
di: Kovačević, Filip, et al.
Pubblicazione: (2026)

Refining Covariance Matrix Estimation in Stochastic Gradient Descent Through Bias Reduction
di: Wei, Ziyang, et al.
Pubblicazione: (2026)

Gradient Descent, Stochastic Optimization, and Other Tales
di: Lu, Jun
Pubblicazione: (2022)

Online Statistical Inference for Contextual Bandits via Stochastic Gradient Descent
di: Chang, Xiangyu, et al.
Pubblicazione: (2022)

Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults
di: Phunyaphibarn, Prin, et al.
Pubblicazione: (2023)

Stochastic Adaptive Gradient Descent Without Descent
di: Aujol, Jean-François, et al.
Pubblicazione: (2025)

Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent
di: Ziyin, Liu, et al.
Pubblicazione: (2024)

A Theoretical Analysis of Noise Geometry in Stochastic Gradient Descent
di: Wang, Mingze, et al.
Pubblicazione: (2023)

Adaptive Batch Size and Learning Rate Scheduler for Stochastic Gradient Descent Based on Minimization of Stochastic First-order Oracle Complexity
di: Umeda, Hikaru, et al.
Pubblicazione: (2025)

Compressed Decentralized Momentum Stochastic Gradient Methods for Nonconvex Optimization
di: Liu, Wei, et al.
Pubblicazione: (2025)

Improving Energy Natural Gradient Descent through Woodbury, Momentum, and Randomization
di: Guzmán-Cordero, Andrés, et al.
Pubblicazione: (2025)

Robust Gradient Descent via Heavy-Ball Momentum with Predictive Extrapolation
di: Ali, Sarwan
Pubblicazione: (2025)

AGGC: Adaptive Group Gradient Clipping for Stabilizing Large Language Model Training
di: Li, Zhiyuan, et al.
Pubblicazione: (2026)

Distributed Gradient Descent for Functional Learning
di: Yu, Zhan, et al.
Pubblicazione: (2023)

Adjacent Leader Decentralized Stochastic Gradient Descent
di: He, Haoze, et al.
Pubblicazione: (2024)

A Bootstrap Perspective on Stochastic Gradient Descent
di: Lan, Hongjian, et al.
Pubblicazione: (2025)

Stochastic Gradient Descent for Nonparametric Additive Regression
di: Chen, Xin, et al.
Pubblicazione: (2024)

Bolstering Stochastic Gradient Descent with Model Building
di: Birbil, S. Ilker, et al.
Pubblicazione: (2021)