Copertina

Gradient correlation is a key ingredient to accelerate SGD with momentum

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Hermant, Julien, Renaud, Marien, Aujol, Jean-François, Dossal, Charles, Rondepierre, Aude
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Optimization and Control
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Study of the behaviour of Nesterov Accelerated Gradient in a non convex setting: the strongly quasar convex case
di: Hermant, Julien, et al.
Pubblicazione: (2024)

Continuized Nesterov Acceleration for Non-Convex Optimization
di: Hermant, Julien, et al.
Pubblicazione: (2025)

Continuized Nesterov Momentum Achieves the $O(\varepsilon^{-7/4})$ Complexity without Additional Mechanisms
di: Hermant, Julien, et al.
Pubblicazione: (2026)

Strong Convergence of FISTA Iterates under H{ö}lderian and Quadratic Growth Conditions
di: Aujol, Jean-François, et al.
Pubblicazione: (2024)

Heavy Ball Momentum for Non-Strongly Convex Optimization
di: Aujol, Jean-François, et al.
Pubblicazione: (2024)

Acceleration for Polyak-Łojasiewicz Functions with a Gradient Aiming Condition
di: Hermant, Julien
Pubblicazione: (2026)

Stochastic Adaptive Gradient Descent Without Descent
di: Aujol, Jean-François, et al.
Pubblicazione: (2025)

Optimization with First Order Algorithms
di: Dossal, Charles, et al.
Pubblicazione: (2024)

Projected Block Coordinate Descent for sparse spike estimation
di: Bénard, Pierre-Jean, et al.
Pubblicazione: (2024)

On the Moreau envelope properties of weakly convex functions
di: Renaud, Marien, et al.
Pubblicazione: (2025)

A note on the convergence of RED algorithms under minimal hypotheses and open questions
di: Traonmilin, Yann, et al.
Pubblicazione: (2026)

From sparse recovery to plug-and-play priors, understanding trade-offs for stable recovery with generalized projected gradient descent
di: Joundi, Ali, et al.
Pubblicazione: (2025)

Optimality of a refraction strategy in the optimal dividends problem with absolutely continuous controls subject to Parisian ruin
di: Locas, Félix, et al.
Pubblicazione: (2023)

SGD with memory: fundamental properties and stochastic acceleration
di: Yarotsky, Dmitry, et al.
Pubblicazione: (2024)

Demystifying SGD with Doubly Stochastic Gradients
di: Kim, Kyurae, et al.
Pubblicazione: (2024)

Dynamics of SGD with Stochastic Polyak Stepsizes: Truly Adaptive Variants and Convergence to Exact Solution
di: Orvieto, Antonio, et al.
Pubblicazione: (2022)

Convergence of SGD with momentum in the nonconvex case: A time window-based analysis
di: Qiu, Junwen, et al.
Pubblicazione: (2024)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
di: Xie, Shengping, et al.
Pubblicazione: (2025)

From Gradient Clipping to Normalization for Heavy Tailed SGD
di: Hübler, Florian, et al.
Pubblicazione: (2024)

Convergence Analysis of a Proximal Stochastic Denoising Regularization Algorithm
di: Renaud, Marien, et al.
Pubblicazione: (2024)

GPU-accelerated dynamic nonlinear optimization with ExaModels and MadNLP
di: Pacaud, François, et al.
Pubblicazione: (2024)

AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent
di: Surjanovic, Nikola, et al.
Pubblicazione: (2025)

From Inexact Gradients to Byzantine Robustness: Acceleration and Optimization under Similarity
di: Gaucher, Renaud, et al.
Pubblicazione: (2026)

On the Set of Possible Minimizers of a Sum of Convex Functions
di: Zamani, Moslem, et al.
Pubblicazione: (2024)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
di: Sun, Tao, et al.
Pubblicazione: (2024)

Local SGD and Federated Averaging Through the Lens of Time Complexity
di: Fradin, Adrien, et al.
Pubblicazione: (2025)

Last-Iterate Complexity of SGD for Convex and Smooth Stochastic Problems
di: Garrigos, Guillaume, et al.
Pubblicazione: (2025)

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
di: Dahan, Tehila, et al.
Pubblicazione: (2023)

Making SGD Parameter-Free
di: Carmon, Yair, et al.
Pubblicazione: (2022)

On the Trajectories of SGD Without Replacement
di: Beneventano, Pierfrancesco
Pubblicazione: (2023)

Convergence and Inference of Stream SGD, with Applications to Queueing Systems and Inventory Control
di: Li, Xiang, et al.
Pubblicazione: (2023)

Avoiding Bias in Clipped SGD for Overparameterized Models under Generalized Smoothness
di: Lobanov, Aleksandr, et al.
Pubblicazione: (2026)

Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithms
di: Mohammadi, Hesameddin, et al.
Pubblicazione: (2022)

An accelerated first-order regularized momentum descent ascent algorithm for stochastic nonconvex-concave minimax problems
di: Zhang, Huiling, et al.
Pubblicazione: (2023)

Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
di: Tyurin, Alexander, et al.
Pubblicazione: (2024)

Inertial Bregman Proximal Gradient under Partial Smoothness
di: Godeme, Jean-Jacques
Pubblicazione: (2025)

Optimality of a barrier strategy in a spectrally negative Lévy model with a level-dependent intensity of bankruptcy
di: Mata, Dante, et al.
Pubblicazione: (2024)

Learning-rate-free Momentum SGD with Reshuffling Converges in Nonsmooth Nonconvex Optimization
di: Hu, Xiaoyin, et al.
Pubblicazione: (2024)

Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions
di: Gaash, Ofir, et al.
Pubblicazione: (2025)

Dimension-adapted Momentum Outscales SGD
di: Ferbach, Damien, et al.
Pubblicazione: (2025)