Cover Image

SNOO: Step-K Nesterov Outer Optimizer - The Surprising Effectiveness of Nesterov Momentum Applied to Pseudo-Gradients

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kallusky, Dominik, Rao, Vinay, Nandavanam, Vishal, Shi, Hao-Jun Michael
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

YuriiFormer: A Suite of Nesterov-Accelerated Transformers
by: Zimin, Aleksandr, et al.
Published: (2026)

Enhanced NIRMAL Optimizer With Damped Nesterov Acceleration: A Comparative Analysis
by: Gaud, Nirmal, et al.
Published: (2025)

Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks
by: Liu, Xin, et al.
Published: (2022)

Accelerated Policy Gradient: On the Convergence Rates of the Nesterov Momentum for Reinforcement Learning
by: Chen, Yen-Ju, et al.
Published: (2023)

EMA-Nesterov: Stabilizing Nesterov's Lookahead for Accelerated Deep Learning Optimization
by: Yau, Chung-Yiu, et al.
Published: (2026)

Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
by: Xie, Xingyu, et al.
Published: (2022)

On Surprising Effectiveness of Masking Updates in Adaptive Optimizers
by: Joo, Taejong, et al.
Published: (2026)

A Concise Lyapunov Analysis of Nesterov's Accelerated Gradient Method
by: Liu, Jun
Published: (2025)

Randomized Subspace Nesterov Accelerated Gradient
by: Omiya, Gaku, et al.
Published: (2026)

Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimization
by: Borodich, Ekaterina, et al.
Published: (2025)

The Surprising Effectiveness of Skip-Tuning in Diffusion Sampling
by: Ma, Jiajun, et al.
Published: (2024)

Nesterov Method for Asynchronous Pipeline Parallel Optimization
by: Ajanthan, Thalaiyasingam, et al.
Published: (2025)

Provable Accelerated Convergence of Nesterov's Momentum for Deep ReLU Neural Networks
by: Liao, Fangshuo, et al.
Published: (2023)

Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
by: Choudhury, Sayantan, et al.
Published: (2026)

Generalized Continuous-Time Models for Nesterov's Accelerated Gradient Methods
by: Park, Chanwoong, et al.
Published: (2024)

The Surprising Effectiveness of Rankers Trained on Expanded Queries
by: Anand, Abhijit, et al.
Published: (2024)

Revisiting the Initial Steps in Adaptive Gradient Descent Optimization
by: Abuduweili, Abulikemu, et al.
Published: (2024)

Take a Step and Reconsider: Sequence Decoding for Self-Improved Neural Combinatorial Optimization
by: Pirnay, Jonathan, et al.
Published: (2024)

On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling
by: Haas, Moritz, et al.
Published: (2025)

Stochastic Gradient Descent with Momentum is Algorithmically Stable
by: Lei, Yunwen, et al.
Published: (2026)

Improving Infinitely Deep Bayesian Neural Networks with Nesterov's Accelerated Gradient Method
by: Yu, Chenxu, et al.
Published: (2026)

The Surprising Effectiveness of Test-Time Training for Few-Shot Learning
by: Akyürek, Ekin, et al.
Published: (2024)

Distributed Low-Communication Training with Decoupled Momentum Optimization
by: Nedelkoski, Sasho, et al.
Published: (2025)

Nesterov acceleration in benignly non-convex landscapes
by: Gupta, Kanan, et al.
Published: (2024)

Nesterov Acceleration for Ensemble Kalman Inversion and Variants
by: Vernon, Sydney, et al.
Published: (2025)

MONA: Muon Optimizer with Nesterov Acceleration for Scalable Language Model Training
by: Li, Jiacheng, et al.
Published: (2026)

DeMo: Decoupled Momentum Optimization
by: Peng, Bowen, et al.
Published: (2024)

On the Limits of Momentum in Decentralized and Federated Optimization
by: Zaccone, Riccardo, et al.
Published: (2025)

Nesterov acceleration despite very noisy gradients
by: Gupta, Kanan, et al.
Published: (2023)

Double Momentum Method for Lower-Level Constrained Bilevel Optimization
by: Shi, Wanli, et al.
Published: (2024)

Offline Model-Based Optimization via Policy-Guided Gradient Search
by: Chemingui, Yassine, et al.
Published: (2024)

Provable Acceleration of Nesterov's Accelerated Gradient for Rectangular Matrix Factorization and Linear Neural Networks
by: Xu, Zhenghao, et al.
Published: (2024)

Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over Larger Language Models
by: Setty, Vinay
Published: (2024)

Learning Surrogates for Offline Black-Box Optimization via Gradient Matching
by: Hoang, Minh, et al.
Published: (2025)

Gradient Dynamics of Attention: How Cross-Entropy Sculpts Bayesian Manifolds
by: Agarwal, Naman, et al.
Published: (2025)

Distributed Nesterov Flows for Multi-agent Optimization
by: Ren, Zihao, et al.
Published: (2026)

FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuning
by: Yan, Peishen, et al.
Published: (2026)

Surprisal Driven $k$-NN for Robust and Interpretable Nonparametric Learning
by: Banerjee, Amartya, et al.
Published: (2023)

Inference of Online Newton Methods with Nesterov's Accelerated Sketching
by: Wang, Haoxuan, et al.
Published: (2026)

Accelerating LMO-Based Optimization via Implicit Gradient Transport
by: Jang, Won-Jun, et al.
Published: (2026)