Image de couverture de livre

Convergence of Distributed Adaptive Optimization with Local Updates

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Cheng, Ziheng, Glasgow, Margalit
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
par: Patel, Kumar Kshitij, et autres
Publié: (2024)

On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
par: Zhou, Dongruo, et autres
Publié: (2018)

Convergence for Discrete Parameter Update Schemes
par: Wilson, Paul, et autres
Publié: (2025)

Local Linear Convergence of Infeasible Optimization with Orthogonal Constraints
par: Sun, Youbang, et autres
Publié: (2024)

Drop-Muon: Update Less, Converge Faster
par: Gruntkowska, Kaja, et autres
Publié: (2025)

Convergence Analysis of the Lion Optimizer in Centralized and Distributed Settings
par: Jiang, Wei, et autres
Publié: (2025)

Adaptive Algorithms with Sharp Convergence Rates for Stochastic Hierarchical Optimization
par: Gong, Xiaochuan, et autres
Publié: (2025)

Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimator
par: Guo, Zhishuai, et autres
Publié: (2021)

Adam Converges Without Any Modification On Update Rules
par: Zhang, Yushun, et autres
Publié: (2026)

Weak Convergence Analysis of Online Neural Actor-Critic Algorithms
par: Lam, Samuel Chun-Hei, et autres
Publié: (2024)

Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates
par: Zhang, Siqi, et autres
Publié: (2023)

Less is More: Convergence Benefits of Fewer Data Weight Updates over Longer Horizon
par: Das, Rudrajit, et autres
Publié: (2026)

Adaptive Delayed-Update Cyclic Algorithm for Variational Inequalities
par: Wei, Yi, et autres
Publié: (2026)

Directional Smoothness and Gradient Methods: Convergence and Adaptivity
par: Mishkin, Aaron, et autres
Publié: (2024)

Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods
par: Lau, Tim Tsz-Kit, et autres
Publié: (2024)

An Efficient Global Optimization Algorithm with Adaptive Estimates of the Local Lipschitz Constants
par: D'Agostino, Danny
Publié: (2022)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
par: Xie, Shengping, et autres
Publié: (2025)

Achieving Near-Optimal Convergence for Distributed Minimax Optimization with Adaptive Stepsizes
par: Huang, Yan, et autres
Publié: (2024)

Fundamental Benefit of Alternating Updates in Minimax Optimization
par: Lee, Jaewook, et autres
Publié: (2024)

Improved Convergence Rates of Muon Optimizer for Nonconvex Optimization
par: Nagashima, Shuntaro, et autres
Publié: (2026)

AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates
par: Zhang, Minxin, et autres
Publié: (2025)

The Effectiveness of Local Updates for Decentralized Learning under Data Heterogeneity
par: Wu, Tongle, et autres
Publié: (2024)

Faster Convergence of Local SGD for Over-Parameterized Models
par: Qin, Tiancheng, et autres
Publié: (2022)

Momentum Benefits Non-IID Federated Learning Simply and Provably
par: Cheng, Ziheng, et autres
Publié: (2023)

On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization
par: Sohrabi, Motahareh, et autres
Publié: (2024)

Automatic Differentiation of Optimization Algorithms with Time-Varying Updates
par: Mehmood, Sheheryar, et autres
Publié: (2024)

TAMUNA: Doubly Accelerated Distributed Optimization with Local Training, Compression, and Partial Participation
par: Condat, Laurent, et autres
Publié: (2023)

Tractable Representations for Convergent Approximation of Distributional HJB Equations
par: Alhosh, Julie, et autres
Publié: (2025)

Convergence of Spectral Descent for Non-smooth Optimization
par: Yang, Yixuan, et autres
Publié: (2026)

Sketch-and-Project Meets Newton Method: Global $\mathcal O(k^{-2})$ Convergence with Low-Rank Updates
par: Hanzely, Slavomír
Publié: (2023)

Optimal Local Convergence Rates of Stochastic First-Order Methods under Local $α$-PL
par: Masiha, Saeed, et autres
Publié: (2024)

Enhancing GNNs Performance on Combinatorial Optimization by Recurrent Feature Update
par: Pugacheva, Daria, et autres
Publié: (2024)

Constant Stepsize Q-learning: Distributional Convergence, Bias and Extrapolation
par: Zhang, Yixuan, et autres
Publié: (2024)

Stochastic Compositional Minimax Optimization with Provable Convergence Guarantees
par: Deng, Yuyang, et autres
Publié: (2024)

On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions
par: Hong, Yusu, et autres
Publié: (2024)

Subspace Optimization for Large Language Models with Convergence Guarantees
par: He, Yutong, et autres
Publié: (2024)

Analyzing and Enhancing the Backward-Pass Convergence of Unrolled Optimization
par: Kotary, James, et autres
Publié: (2023)

Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees
par: Xiao, Nachuan, et autres
Publié: (2023)

Locally Adaptive Federated Learning
par: Mukherjee, Sohom, et autres
Publié: (2023)

Achieving Better Local Regret Bound for Online Non-Convex Bilevel Optimization
par: Jia, Tingkai, et autres
Publié: (2026)