Copertina

Demystifying SGD with Doubly Stochastic Gradients

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Kim, Kyurae, Ko, Joohwan, Ma, Yi-An, Gardner, Jacob R.
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Optimization and Control
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Space
di: Kim, Kyurae, et al.
Pubblicazione: (2026)

Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inference
di: Kim, Kyurae, et al.
Pubblicazione: (2025)

Stochastic Approximation with Biased MCMC for Expectation Maximization
di: Gruffaz, Samuel, et al.
Pubblicazione: (2024)

On the Convergence of Black-Box Variational Inference
di: Kim, Kyurae, et al.
Pubblicazione: (2023)

AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent
di: Surjanovic, Nikola, et al.
Pubblicazione: (2025)

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
di: Dahan, Tehila, et al.
Pubblicazione: (2023)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
di: Xie, Shengping, et al.
Pubblicazione: (2025)

From Gradient Clipping to Normalization for Heavy Tailed SGD
di: Hübler, Florian, et al.
Pubblicazione: (2024)

Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD
di: Andreyev, Arseniy, et al.
Pubblicazione: (2024)

On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
di: Sahu, Sharan, et al.
Pubblicazione: (2026)

A Hessian-Aware Stochastic Differential Equation for Modelling SGD
di: Li, Xiang, et al.
Pubblicazione: (2024)

SketchySGD: Reliable Stochastic Optimization via Randomized Curvature Estimates
di: Frangella, Zachary, et al.
Pubblicazione: (2022)

Provably Scalable Black-Box Variational Inference with Structured Variational Families
di: Ko, Joohwan, et al.
Pubblicazione: (2024)

Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsize
di: Li, Chris Junchi
Pubblicazione: (2024)

Does Worst-Performing Agent Lead the Pack? Analyzing Agent Dynamics in Unified Distributed SGD
di: Hu, Jie, et al.
Pubblicazione: (2024)

SGD at the Edge of Stability: The Stochastic Sharpness Gap
di: Liao, Fangshuo, et al.
Pubblicazione: (2026)

$μ^2$-SGD: Stable Stochastic Optimization via a Double Momentum Mechanism
di: Dahan, Tehila, et al.
Pubblicazione: (2023)

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
di: Yu, Dingzhi, et al.
Pubblicazione: (2026)

Natural Policy Gradient as Doubly Smoothed Policy Iteration: A Bellman-Operator Framework
di: Nanda, Phalguni, et al.
Pubblicazione: (2026)

VAMO: Efficient Zeroth-Order Variance Reduction for SGD with Faster Convergence
di: Chen, Jiahe, et al.
Pubblicazione: (2025)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
di: Sun, Tao, et al.
Pubblicazione: (2024)

Scaling Laws of SignSGD in Linear Regression: When Does It Outperform SGD?
di: Kim, Jihwan, et al.
Pubblicazione: (2026)

Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning
di: Ding, Kuangyu, et al.
Pubblicazione: (2023)

Stochastic Gradients under Nuisances
di: Yu, Facheng, et al.
Pubblicazione: (2025)

Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
di: Jiang, Ruichen, et al.
Pubblicazione: (2024)

Adaptive Gradient Normalization and Independent Sampling for (Stochastic) Generalized-Smooth Optimization
di: Yang, Yufeng, et al.
Pubblicazione: (2024)

Making SGD Parameter-Free
di: Carmon, Yair, et al.
Pubblicazione: (2022)

On the Trajectories of SGD Without Replacement
di: Beneventano, Pierfrancesco
Pubblicazione: (2023)

Convergence of SGD with momentum in the nonconvex case: A time window-based analysis
di: Qiu, Junwen, et al.
Pubblicazione: (2024)

Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent
di: Sato, Naoki, et al.
Pubblicazione: (2024)

Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
di: Tyurin, Alexander, et al.
Pubblicazione: (2024)

Stochastic Gradient Descent with Adaptive Data
di: Che, Ethan, et al.
Pubblicazione: (2024)

Bilevel Learning with Inexact Stochastic Gradients
di: Salehi, Mohammad Sadegh, et al.
Pubblicazione: (2024)

Stochastic Gradient Methods with Preconditioned Updates
di: Sadiev, Abdurakhmon, et al.
Pubblicazione: (2022)

Stochastic Gradient Descent with Strategic Querying
di: Jiang, Nanfei, et al.
Pubblicazione: (2025)

Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGD
di: Wang, Jiayi, et al.
Pubblicazione: (2020)

Dimension-adapted Momentum Outscales SGD
di: Ferbach, Damien, et al.
Pubblicazione: (2025)

Heavy-Tail Phenomenon in Decentralized SGD
di: Gurbuzbalaban, Mert, et al.
Pubblicazione: (2022)

Derivatives of Stochastic Gradient Descent in parametric optimization
di: Iutzeler, Franck, et al.
Pubblicazione: (2024)

Stochastic Gradient Langevin Dynamics with Variance Reduction
di: Huang, Zhishen, et al.
Pubblicazione: (2021)