Cover Image

VAMO: Efficient Zeroth-Order Variance Reduction for SGD with Faster Convergence

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Jiahe, Ma, Ziye
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Faster Convergence of Local SGD for Over-Parameterized Models
by: Qin, Tiancheng, et al.
Published: (2022)

Muon is Provably Faster with Momentum Variance Reduction
by: Qian, Xun, et al.
Published: (2025)

Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction
by: Jiang, Wei, et al.
Published: (2024)

Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations
by: Ma, Shaocong, et al.
Published: (2025)

Drago: Primal-Dual Coupled Variance Reduction for Faster Distributionally Robust Optimization
by: Mehta, Ronak, et al.
Published: (2024)

Global Convergence of Natural Policy Gradient with Hessian-aided Momentum Variance Reduction
by: Feng, Jie, et al.
Published: (2024)

On Adaptivity in Zeroth-Order Optimization
by: Dbouk, Hassan, et al.
Published: (2026)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
by: Xie, Shengping, et al.
Published: (2025)

Convergence of SGD with momentum in the nonconvex case: A time window-based analysis
by: Qiu, Junwen, et al.
Published: (2024)

Zeroth-Order Methods for Stochastic Nonconvex Nonsmooth Composite Optimization
by: Chen, Ziyi, et al.
Published: (2025)

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
by: Gautam, Tanmay, et al.
Published: (2024)

Riemannian Zeroth-Order Gradient Estimation with Structure-Preserving Metrics for Geodesically Incomplete Manifolds
by: Ma, Shaocong, et al.
Published: (2026)

Unlocking TriLevel Learning with Level-Wise Zeroth Order Constraints: Distributed Algorithms and Provable Non-Asymptotic Convergence
by: Jiao, Yang, et al.
Published: (2024)

SGD for Variational Inference: Tackling Unbounded Variance via Preconditioning and Dynamic Batching
by: Labarrière, Hippolyte, et al.
Published: (2026)

Global Convergence of SGD On Two Layer Neural Nets
by: Gopalani, Pulkit, et al.
Published: (2022)

On the Inherent Privacy of Zeroth Order Projected Gradient Descent
by: Gupta, Devansh, et al.
Published: (2025)

Lower Bounds and Proximally Anchored SGD for Non-Convex Minimization Under Unbounded Variance
by: Fazla, Arda, et al.
Published: (2026)

Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime
by: Attia, Amit, et al.
Published: (2025)

On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization
by: Ma, Shaocong, et al.
Published: (2025)

Zeroth-Order Hard-Thresholding: Gradient Error vs. Expansivity
by: de Vazelhes, William, et al.
Published: (2022)

Fully Zeroth-Order Bilevel Programming via Gaussian Smoothing
by: Aghasi, Alireza, et al.
Published: (2024)

ZIVR: An Incremental Variance Reduction Technique For Zeroth-Order Composite Problems
by: Zhang, Silan, et al.
Published: (2026)

A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD
by: Jin, Ruinan, et al.
Published: (2024)

Global Convergence of SGD For Logistic Loss on Two Layer Neural Nets
by: Gopalani, Pulkit, et al.
Published: (2023)

LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM
by: Refael, Yehonathan, et al.
Published: (2025)

On the Convergence of DP-SGD with Adaptive Clipping
by: Shulgin, Egor, et al.
Published: (2024)

Drop-Muon: Update Less, Converge Faster
by: Gruntkowska, Kaja, et al.
Published: (2025)

A Randomized Zeroth-Order Hierarchical Framework for Heterogeneous Federated Learning
by: Qiu, Yuyang, et al.
Published: (2025)

Minimisation of Polyak-Łojasewicz Functions Using Random Zeroth-Order Oracles
by: Farzin, Amir Ali, et al.
Published: (2024)

Zeroth-Order Optimization at the Edge of Stability
by: Song, Minhak, et al.
Published: (2026)

Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient
by: Di, Hao, et al.
Published: (2024)

Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
by: Khah, Saleh Vatan, et al.
Published: (2025)

Adaptive SGD with Line-Search and Polyak Stepsizes: Nonconvex Convergence and Accelerated Rates
by: Wu, Haotian
Published: (2025)

Convergence and concentration properties of constant step-size SGD through Markov chains
by: Merad, Ibrahim, et al.
Published: (2023)

Zeroth-Order Stochastic Mirror Descent Algorithms for Minimax Excess Risk Optimization
by: Gu, Zhihao, et al.
Published: (2024)

Diagonalisation SGD: Fast & Convergent SGD for Non-Differentiable Models via Reparameterisation and Smoothing
by: Wagner, Dominik, et al.
Published: (2024)

Convergence of SGD for Training Neural Networks with Sliced Wasserstein Losses
by: Tanguy, Eloi
Published: (2023)

High-Probability Convergence Guarantees of Decentralized SGD
by: Armacki, Aleksandar, et al.
Published: (2025)

Zeroth-Order Optimization Finds Flat Minima
by: Zhang, Liang, et al.
Published: (2025)

Certified Multi-Fidelity Zeroth-Order Optimization
by: de Montbrun, Étienne, et al.
Published: (2023)