Image de couverture de livre

Momentum Benefits Non-IID Federated Learning Simply and Provably

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Cheng, Ziheng, Huang, Xinmeng, Wu, Pengfei, Yuan, Kun
Format:	Preprint
Publié:	2023
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Provably Convergent Federated Trilevel Learning
par: Jiao, Yang, et autres
Publié: (2023)

Provable Reduction in Communication Rounds for Non-Smooth Convex Federated Learning
par: Palenzuela, Karlo, et autres
Publié: (2025)

Muon is Provably Faster with Momentum Variance Reduction
par: Qian, Xun, et autres
Publié: (2025)

SPARKLE: A Unified Single-Loop Primal-Dual Framework for Decentralized Bilevel Optimization
par: Zhu, Shuchen, et autres
Publié: (2024)

Decentralized Bilevel Optimization: A Perspective from Transient Iteration Complexity
par: Kong, Boao, et autres
Publié: (2024)

On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
par: Sahu, Sharan, et autres
Publié: (2026)

Unbiased Compression Saves Communication in Distributed Optimization: When and How Much?
par: He, Yutong, et autres
Publié: (2023)

Stochastic Controlled Averaging for Federated Learning with Communication Compression
par: Huang, Xinmeng, et autres
Publié: (2023)

Provable Accelerated Convergence of Nesterov's Momentum for Deep ReLU Neural Networks
par: Liao, Fangshuo, et autres
Publié: (2023)

Non-convex Stochastic Composite Optimization with Polyak Momentum
par: Gao, Yuan, et autres
Publié: (2024)

FedSEA: Achieving Benefit of Parallelization in Federated Online Learning
par: Sahu, Harekrushna, et autres
Publié: (2026)

Achieving Linear Speedup for Composite Federated Learning
par: Huang, Kun, et autres
Publié: (2026)

Convergence of Distributed Adaptive Optimization with Local Updates
par: Cheng, Ziheng, et autres
Publié: (2024)

Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation
par: Otsuka, Kaoru, et autres
Publié: (2025)

Random Scaling and Momentum for Non-smooth Non-convex Optimization
par: Zhang, Qinzi, et autres
Publié: (2024)

Provable Adaptivity of Adam under Non-uniform Smoothness
par: Wang, Bohan, et autres
Publié: (2022)

Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression
par: He, Yutong, et autres
Publié: (2023)

Beyond IID: data-driven decision-making in heterogeneous environments
par: Besbes, Omar, et autres
Publié: (2022)

Towards Simple and Provable Parameter-Free Adaptive Gradient Methods
par: Tao, Yuanzhe, et autres
Publié: (2024)

Learning Provably Improves the Convergence of Gradient Descent
par: Song, Qingyu, et autres
Publié: (2025)

Provable Mixed-Noise Learning with Flow-Matching
par: Hagemann, Paul, et autres
Publié: (2025)

Unlocking TriLevel Learning with Level-Wise Zeroth Order Constraints: Distributed Algorithms and Provable Non-Asymptotic Convergence
par: Jiao, Yang, et autres
Publié: (2024)

Provable Exactness for Asymmetric Low-Rank SDP Learning
par: Hu, Enliang
Publié: (2018)

Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum
par: Zhou, Yuan, et autres
Publié: (2025)

Momentum for the Win: Collaborative Federated Reinforcement Learning across Heterogeneous Environments
par: Wang, Han, et autres
Publié: (2024)

Provably Faster Algorithms for Bilevel Optimization via Without-Replacement Sampling
par: Li, Junyi, et autres
Publié: (2024)

One-Layer Transformer Provably Learns One-Nearest Neighbor In Context
par: Li, Zihao, et autres
Publié: (2024)

On The Concurrence of Layer-wise Preconditioning Methods and Provable Feature Learning
par: Zhang, Thomas T., et autres
Publié: (2025)

Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent
par: Chu, Ya-Chi, et autres
Publié: (2025)

Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State
par: Cheng, Ziheng, et autres
Publié: (2025)

The Marginal Value of Momentum for Small Learning Rate SGD
par: Wang, Runzhe, et autres
Publié: (2023)

Agentic Transformers Provably Learn to Search via Reinforcement Learning
par: Yang, Tong, et autres
Publié: (2026)

SPAM: Stochastic Proximal Point Method with Momentum Variance Reduction for Non-convex Cross-Device Federated Learning
par: Karagulyan, Avetik, et autres
Publié: (2024)

Both Asymptotic and Non-Asymptotic Convergence of Quasi-Hyperbolic Momentum using Increasing Batch Size
par: Imaizumi, Kento, et autres
Publié: (2025)

Guided by the Experts: Provable Feature Learning Dynamic of Soft-Routed Mixture-of-Experts
par: Liao, Fangshuo, et autres
Publié: (2025)

Traversing Pareto Optimal Policies: Provably Efficient Multi-Objective Reinforcement Learning
par: Qiu, Shuang, et autres
Publié: (2024)

Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
par: Jiang, Ruichen, et autres
Publié: (2024)

Provably Efficient Exploration in Policy Optimization
par: Cai, Qi, et autres
Publié: (2019)

Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Acceleration
par: Khaled, Ahmed, et autres
Publié: (2025)

Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
par: Choudhury, Sayantan, et autres
Publié: (2026)